2023-06-27T10:46:31.763 INFO:root:teuthology version: 0.0.1.dev130+g3fdf1f2 2023-06-27T10:46:31.765 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2023-06-27_10:34:13-crimson-rados-wip-matanb-crimson-61651-distro-crimson-smithi/7317795 --description crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-localized} --name matan-2023-06-27_10:34:13-crimson-rados-wip-matanb-crimson-61651-distro-crimson-smithi -- /home/teuthworker/archive/matan-2023-06-27_10:34:13-crimson-rados-wip-matanb-crimson-61651-distro-crimson-smithi/7317795/orig.config.yaml 2023-06-27T10:46:31.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-27T10:46:31.886 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2023-06-27_10:34:13-crimson-rados-wip-matanb-crimson-61651-distro-crimson-smithi/7317795 branch: wip-matanb-crimson-61651 description: crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-localized} email: null first_in_suite: false job_id: '7317795' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: matan-2023-06-27_10:34:13-crimson-rados-wip-matanb-crimson-61651-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-matanb-crimson-61651 ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 osd pool default crimson: true paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore rocksdb cf: false crimson seastar smp: 3 debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 err to stderr: true log flush on exit: true log to file: false log to stderr: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: bluestore osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: crimson fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: ba3fff7216d65a7042e2ef64fe924764dc57a097 wait-for-scrub: false ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: crimson sha1: ba3fff7216d65a7042e2ef64fe924764dc57a097 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: min_in: 4 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: fd62a8dfcb345fabed3eea6b3a461f1ca848a239 owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b sha1: ba3fff7216d65a7042e2ef64fe924764dc57a097 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: fd62a8dfcb345fabed3eea6b3a461f1ca848a239 targets: smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgThrJDXz65+FadHewTpcJkFYPfdgbxe7nQSh+DEDbi4mDOvkqyFOc3bi6SDKEqmTapP1If/P6toZyfU4STTzvBqlhh2il/4OROft9mllriUT9jcBaoO5km8c+uqV1q4mKRNhV1zEBE8lKageTV0j9OD54nnStOMHYOpxHhWBVmWnb/OuatqDFxHgmiLPsPkVCx1TU8QPDjEP9yzmK92xOVQ53XpvohTtuuhVl998uflK4iCbSNoFsMawVl2m34XbbQwSAm0Ar0hvUTK5Oa8LAJ81D8ChQjUaMzqc/r0/Me+iB1ScMpLdBeCOrcSGYE7Gt1xBqJ09bA21WVYxSdtmjXHiOZ0AaXnh5dYnz1U5DNgFsZgZsBLrukhDLXNWmzfZuW6K/d1KTMZtKBiqwrpFEwpYto7RyOQRY4VDsm+NrufI+Zt/hXJXW4UFLWT/0HcO5+AHk1IC4WI+zCGKEgw6EfpVo2CJYGPUGsiqR8yPJGye4GSgys4glyvdIwLAPcXE= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJT0rSA4ew9YllXsd9mCkiTOfvJut+5MH8gcgMh43A7RxIsTObiUnvZcaZH0xL72eEuprrxVs6iVsDdlSby1cY= tasks: - install: null - ceph: conf: osd: debug monc: 20 flavor: crimson - thrashosds: ceph_objectstore_tool: false chance_down: 0 chance_inject_pause_short: 0 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 dump_ops_enable: false min_in: 3 noscrub_toggle_delay: 0 reweight_osd: 0 sighup_delay: 0 thrash_primary_affinity: false timeout: 2400 - rados: clients: - client.0 localize_reads: true objects: 50 op_weights: copy_from: 0 delete: 50 read: 100 rollback: 0 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3fdf1f2d90b4263fafe8c0792229f37d37a2ab95 timestamp: 2023-06-27_10:34:13 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-27T10:46:31.887 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa; will attempt to use it 2023-06-27T10:46:31.887 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks 2023-06-27T10:46:31.888 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-27T10:46:31.890 INFO:teuthology.task.internal:Checking packages... 2023-06-27T10:46:31.917 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash 'ba3fff7216d65a7042e2ef64fe924764dc57a097' 2023-06-27T10:46:31.917 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-27T10:46:31.918 INFO:teuthology.packaging:ref: None 2023-06-27T10:46:31.918 INFO:teuthology.packaging:tag: None 2023-06-27T10:46:31.918 INFO:teuthology.packaging:branch: wip-matanb-crimson-61651 2023-06-27T10:46:31.919 INFO:teuthology.packaging:sha1: ba3fff7216d65a7042e2ef64fe924764dc57a097 2023-06-27T10:46:31.919 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&ref=wip-matanb-crimson-61651 2023-06-27T10:46:32.137 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4614.gba3fff72 2023-06-27T10:46:32.137 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-27T10:46:32.144 INFO:teuthology.task.internal:no buildpackages task found 2023-06-27T10:46:32.144 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-27T10:46:32.162 INFO:teuthology.task.internal:Saving configuration 2023-06-27T10:46:32.177 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-27T10:46:32.183 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-27T10:46:32.208 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-06-27_10:34:13-crimson-rados-wip-matanb-crimson-61651-distro-crimson-smithi/7317795', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-27 10:41:21.911412', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJllL3qBsZp1C2Re2GRvHOJSpFNDsS7lkXE7xqpM/y82LDzf/LrK947QczwaGelBRho/gCeN5eo1XpLimeSwgjc='} 2023-06-27T10:46:32.227 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-06-27_10:34:13-crimson-rados-wip-matanb-crimson-61651-distro-crimson-smithi/7317795', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-27 10:41:21.913310', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJT0rSA4ew9YllXsd9mCkiTOfvJut+5MH8gcgMh43A7RxIsTObiUnvZcaZH0xL72eEuprrxVs6iVsDdlSby1cY='} 2023-06-27T10:46:32.228 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-27T10:46:32.234 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2023-06-27T10:46:32.234 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-06-27T10:46:32.234 INFO:teuthology.run_tasks:Running task console_log... 2023-06-27T10:46:32.313 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3f8b599ee0>, signals=[15]) 2023-06-27T10:46:32.314 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-27T10:46:32.321 INFO:teuthology.task.internal:Opening connections... 2023-06-27T10:46:32.321 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2023-06-27T10:46:32.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-27T10:46:32.390 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2023-06-27T10:46:32.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-27T10:46:32.457 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-27T10:46:32.468 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2023-06-27T10:46:32.491 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2023-06-27T10:46:32.491 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2023-06-27T10:46:32.551 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2023-06-27T10:46:32.551 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8" 2023-06-27T10:46:32.551 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2023-06-27T10:46:32.551 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2023-06-27T10:46:32.551 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8" 2023-06-27T10:46:32.552 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2023-06-27T10:46:32.552 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-27T10:46:32.552 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2023-06-27T10:46:32.552 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-27T10:46:32.552 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2023-06-27T10:46:32.552 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-27T10:46:32.552 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-27T10:46:32.552 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-27T10:46:32.553 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2023-06-27T10:46:32.587 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2023-06-27T10:46:32.609 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2023-06-27T10:46:32.610 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2023-06-27T10:46:32.670 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2023-06-27T10:46:32.670 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8" 2023-06-27T10:46:32.670 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2023-06-27T10:46:32.670 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2023-06-27T10:46:32.670 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8" 2023-06-27T10:46:32.671 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2023-06-27T10:46:32.671 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-27T10:46:32.671 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2023-06-27T10:46:32.671 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-27T10:46:32.671 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2023-06-27T10:46:32.671 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-27T10:46:32.671 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-27T10:46:32.672 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-27T10:46:32.672 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2023-06-27T10:46:32.703 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-27T10:46:32.712 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-27T10:46:32.716 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-27T10:46:32.717 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2023-06-27T10:46:32.719 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2023-06-27T10:46:32.738 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-27T10:46:32.744 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-27T10:46:32.744 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2023-06-27T10:46:32.780 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2023-06-27T10:46:32.820 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-27T10:46:32.878 INFO:teuthology.run_tasks:Running task kernel... 2023-06-27T10:46:32.892 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-06-27T10:46:32.893 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-06-27T10:46:32.894 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-27T10:46:32.894 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-27T10:46:32.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-27T10:46:32.914 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2023-06-27T10:46:32.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-27T10:46:32.915 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2023-06-27T10:46:32.973 INFO:teuthology.orchestra.run.smithi159.stdout:4.18.0-489.el8.x86_64 2023-06-27T10:46:32.973 INFO:teuthology.task.kernel:Running kernel on smithi159: 4.18.0-489.el8.x86_64 2023-06-27T10:46:32.973 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum install -y kernel 2023-06-27T10:46:32.975 INFO:teuthology.orchestra.run.smithi138.stdout:4.18.0-489.el8.x86_64 2023-06-27T10:46:32.975 INFO:teuthology.task.kernel:Running kernel on smithi138: 4.18.0-489.el8.x86_64 2023-06-27T10:46:32.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2023-06-27T10:46:54.965 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - AppStream 1.6 MB/s | 30 MB 00:19 2023-06-27T10:46:55.024 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 8 - AppStream 1.6 MB/s | 30 MB 00:19 2023-06-27T11:25:06.425 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 8 - BaseOS 18 kB/s | 40 MB 38:05 2023-06-27T11:25:12.615 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 8 - Extras 45 kB/s | 18 kB 00:00 2023-06-27T11:25:13.133 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 8 - Extras common packages 19 kB/s | 6.6 kB 00:00 2023-06-27T11:25:16.009 INFO:teuthology.orchestra.run.smithi159.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-27T11:25:17.093 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-06-27T11:25:20.918 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 173 kB/s | 24 kB 00:00 2023-06-27T11:25:24.134 INFO:teuthology.orchestra.run.smithi159.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-06-27T11:25:24.134 INFO:teuthology.orchestra.run.smithi159.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-06-27T11:25:24.344 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T11:25:24.344 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T11:25:24.345 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2023-06-27T11:25:24.345 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T11:25:24.345 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T11:25:24.345 INFO:teuthology.orchestra.run.smithi159.stdout: kernel x86_64 4.18.0-499.el8 baseos 9.8 M 2023-06-27T11:25:24.345 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-06-27T11:25:24.345 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-core x86_64 4.18.0-499.el8 baseos 43 M 2023-06-27T11:25:24.345 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules x86_64 4.18.0-499.el8 baseos 35 M 2023-06-27T11:25:24.346 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T11:25:24.346 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T11:25:24.346 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T11:25:24.346 INFO:teuthology.orchestra.run.smithi159.stdout:Install 3 Packages 2023-06-27T11:25:24.346 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T11:25:24.347 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 88 M 2023-06-27T11:25:24.347 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 95 M 2023-06-27T11:25:24.347 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T11:40:43.708 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - BaseOS 13 kB/s | 40 MB 53:43 2023-06-27T11:40:50.490 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - Extras 18 kB/s | 18 kB 00:01 2023-06-27T11:40:52.474 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - Extras common packages 3.6 kB/s | 6.6 kB 00:01 2023-06-27T11:40:55.292 INFO:teuthology.orchestra.run.smithi138.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-27T11:40:56.936 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-06-27T11:41:00.742 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 199 kB/s | 24 kB 00:00 2023-06-27T11:41:03.908 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-06-27T11:41:03.908 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-06-27T11:41:04.122 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T11:41:04.122 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T11:41:04.123 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2023-06-27T11:41:04.123 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T11:41:04.123 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T11:41:04.123 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 4.18.0-499.el8 baseos 9.8 M 2023-06-27T11:41:04.123 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-06-27T11:41:04.123 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core x86_64 4.18.0-499.el8 baseos 43 M 2023-06-27T11:41:04.123 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules x86_64 4.18.0-499.el8 baseos 35 M 2023-06-27T11:41:04.124 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T11:41:04.124 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T11:41:04.124 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T11:41:04.124 INFO:teuthology.orchestra.run.smithi138.stdout:Install 3 Packages 2023-06-27T11:41:04.124 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T11:41:04.125 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 88 M 2023-06-27T11:41:04.125 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 95 M 2023-06-27T11:41:04.125 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T11:41:21.457 INFO:teuthology.orchestra.run.smithi159.stdout:(1/3): kernel-4.18.0-499.el8.x86_64.rpm 10 kB/s | 9.8 MB 15:57 2023-06-27T11:56:08.324 INFO:teuthology.orchestra.run.smithi138.stdout:(1/3): kernel-4.18.0-499.el8.x86_64.rpm 11 kB/s | 9.8 MB 15:04 2023-06-27T12:01:57.700 INFO:teuthology.orchestra.run.smithi159.stdout:(2/3): kernel-modules-4.18.0-499.el8.x86_64.rpm 16 kB/s | 35 MB 36:33 2023-06-27T12:06:47.146 INFO:teuthology.orchestra.run.smithi159.stdout:(3/3): kernel-core-4.18.0-499.el8.x86_64.rpm 18 kB/s | 43 MB 41:22 2023-06-27T12:06:47.147 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:06:47.147 INFO:teuthology.orchestra.run.smithi159.stdout:Total 36 kB/s | 88 MB 41:22 2023-06-27T12:06:47.802 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:06:48.229 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:06:48.229 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:06:49.434 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:06:49.465 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:06:52.246 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:06:52.702 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-core-4.18.0-499.el8.x86_64 1/3 2023-06-27T12:06:56.256 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-core-4.18.0-499.el8.x86_64 1/3 2023-06-27T12:06:56.743 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-modules-4.18.0-499.el8.x86_64 2/3 2023-06-27T12:07:00.095 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-modules-4.18.0-499.el8.x86_64 2/3 2023-06-27T12:07:00.437 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:07:21.937 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-core-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:07:21.959 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-modules-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:07:22.298 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:07:22.299 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-4.18.0-499.el8.x86_64 1/3 2023-06-27T12:07:22.299 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-core-4.18.0-499.el8.x86_64 2/3 2023-06-27T12:07:23.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-modules-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:07:23.285 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:07:23.285 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:07:23.285 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-4.18.0-499.el8.x86_64 kernel-core-4.18.0-499.el8.x86_64 2023-06-27T12:07:23.286 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules-4.18.0-499.el8.x86_64 2023-06-27T12:07:23.286 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:07:23.286 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:07:23.421 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q kernel | sort -rV | head -n 1 2023-06-27T12:07:23.633 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-4.18.0-499.el8.x86_64 2023-06-27T12:07:23.633 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-499.el8.x86_64 2023-06-27T12:07:23.634 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-499.el8.x86_64 2023-06-27T12:07:23.634 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-27T12:07:23.634 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-27T12:07:23.634 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2023-06-27T12:07:23.634 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi159.front.sepia.ceph.com, path=None, version=distro) 2023-06-27T12:07:23.635 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q kernel | sort -rV | head -n 1 2023-06-27T12:07:23.840 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-4.18.0-499.el8.x86_64 2023-06-27T12:07:23.840 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-499.el8.x86_64 2023-06-27T12:07:23.841 DEBUG:teuthology.orchestra.run.smithi159:> sudo rpm -qi grub2-tools 2023-06-27T12:07:23.874 INFO:teuthology.orchestra.run.smithi159.stdout:Name : grub2-tools 2023-06-27T12:07:23.874 INFO:teuthology.orchestra.run.smithi159.stdout:Epoch : 1 2023-06-27T12:07:23.875 INFO:teuthology.orchestra.run.smithi159.stdout:Version : 2.02 2023-06-27T12:07:23.875 INFO:teuthology.orchestra.run.smithi159.stdout:Release : 129.el8 2023-06-27T12:07:23.875 INFO:teuthology.orchestra.run.smithi159.stdout:Architecture: x86_64 2023-06-27T12:07:23.875 INFO:teuthology.orchestra.run.smithi159.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-06-27T12:07:23.875 INFO:teuthology.orchestra.run.smithi159.stdout:Group : System Environment/Base 2023-06-27T12:07:23.875 INFO:teuthology.orchestra.run.smithi159.stdout:Size : 9265586 2023-06-27T12:07:23.875 INFO:teuthology.orchestra.run.smithi159.stdout:License : GPLv3+ 2023-06-27T12:07:23.875 INFO:teuthology.orchestra.run.smithi159.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-06-27T12:07:23.876 INFO:teuthology.orchestra.run.smithi159.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-06-27T12:07:23.876 INFO:teuthology.orchestra.run.smithi159.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-06-27T12:07:23.876 INFO:teuthology.orchestra.run.smithi159.stdout:Build Host : kbuilder.bsys.centos.org 2023-06-27T12:07:23.876 INFO:teuthology.orchestra.run.smithi159.stdout:Relocations : (not relocatable) 2023-06-27T12:07:23.876 INFO:teuthology.orchestra.run.smithi159.stdout:Packager : CentOS BuildSystem 2023-06-27T12:07:23.876 INFO:teuthology.orchestra.run.smithi159.stdout:Vendor : CentOS 2023-06-27T12:07:23.876 INFO:teuthology.orchestra.run.smithi159.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-27T12:07:23.877 INFO:teuthology.orchestra.run.smithi159.stdout:Summary : Support tools for GRUB. 2023-06-27T12:07:23.877 INFO:teuthology.orchestra.run.smithi159.stdout:Description : 2023-06-27T12:07:23.877 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:07:23.877 INFO:teuthology.orchestra.run.smithi159.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-27T12:07:23.877 INFO:teuthology.orchestra.run.smithi159.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-27T12:07:23.877 INFO:teuthology.orchestra.run.smithi159.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-27T12:07:23.877 INFO:teuthology.orchestra.run.smithi159.stdout:hardware devices. 2023-06-27T12:07:23.878 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:07:23.878 INFO:teuthology.orchestra.run.smithi159.stdout:This subpackage provides tools for support of all platforms. 2023-06-27T12:07:23.878 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-27T12:07:23.879 INFO:teuthology.task.kernel:Updating grub on smithi159 to boot 4.18.0-499.el8.x86_64 2023-06-27T12:07:23.879 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-27T12:07:25.053 INFO:teuthology.orchestra.run.smithi159.stderr:Generating grub configuration file ... 2023-06-27T12:07:26.898 INFO:teuthology.orchestra.run.smithi159.stderr:done 2023-06-27T12:07:26.900 DEBUG:teuthology.orchestra.run.smithi159:> mktemp 2023-06-27T12:07:26.934 INFO:teuthology.orchestra.run.smithi159.stdout:/tmp/tmp.A9oGIPtNOF 2023-06-27T12:07:26.935 DEBUG:teuthology.orchestra.run.smithi159:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.A9oGIPtNOF 2023-06-27T12:07:27.013 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0666 /tmp/tmp.A9oGIPtNOF 2023-06-27T12:07:27.176 DEBUG:teuthology.orchestra.remote:smithi159:/tmp/tmp.A9oGIPtNOF is 6KB 2023-06-27T12:07:27.224 DEBUG:teuthology.orchestra.run.smithi159:> rm -fr /tmp/tmp.A9oGIPtNOF 2023-06-27T12:07:27.241 DEBUG:teuthology.orchestra.run.smithi159:> sudo /bin/ls /boot/loader/entries || true 2023-06-27T12:07:27.306 INFO:teuthology.orchestra.run.smithi159.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-06-27T12:07:27.306 INFO:teuthology.orchestra.run.smithi159.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-06-27T12:07:27.307 INFO:teuthology.orchestra.run.smithi159.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-06-27T12:07:27.307 INFO:teuthology.orchestra.run.smithi159.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-499.el8.x86_64.conf 2023-06-27T12:07:27.308 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-499.el8.x86_64 2023-06-27T12:07:27.481 DEBUG:teuthology.orchestra.run.smithi159:> sudo shutdown -r now 2023-06-27T12:07:57.484 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2023-06-27T12:07:57.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-06-27T12:07:57.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-27T12:08:15.984 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2023-06-27T12:08:42.884 INFO:teuthology.orchestra.run.smithi138.stdout:(2/3): kernel-core-4.18.0-499.el8.x86_64.rpm 26 kB/s | 43 MB 27:38 2023-06-27T12:08:45.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-06-27T12:08:45.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-27T12:09:04.368 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2023-06-27T12:09:34.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-06-27T12:09:34.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-27T12:09:34.803 DEBUG:teuthology.orchestra.run.smithi159:> true 2023-06-27T12:09:35.062 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-06-27T12:09:35.062 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "4.18.0-499.el8.x86_64"... 2023-06-27T12:09:35.063 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2023-06-27T12:09:35.122 INFO:teuthology.orchestra.run.smithi159.stdout:4.18.0-499.el8.x86_64 2023-06-27T12:09:35.123 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-499.el8.x86_64 vs 4.18.0-499.el8.x86_64 2023-06-27T12:09:35.123 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-27T12:09:35.123 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-06-27T12:09:36.124 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2023-06-27T12:09:36.125 DEBUG:teuthology.orchestra.run.smithi159:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-27T12:09:36.197 INFO:teuthology.orchestra.run.smithi159.stdout:ttyS1 2023-06-27T12:09:36.210 DEBUG:teuthology.parallel:result is None 2023-06-27T12:11:19.587 INFO:teuthology.orchestra.run.smithi138.stdout:(3/3): kernel-modules-4.18.0-499.el8.x86_64.rpm 20 kB/s | 35 MB 30:15 2023-06-27T12:11:19.588 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:11:19.588 INFO:teuthology.orchestra.run.smithi138.stdout:Total 49 kB/s | 88 MB 30:15 2023-06-27T12:11:20.250 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:11:20.698 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:11:20.699 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:11:21.920 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:11:21.944 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:11:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:11:25.443 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-core-4.18.0-499.el8.x86_64 1/3 2023-06-27T12:11:29.014 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-4.18.0-499.el8.x86_64 1/3 2023-06-27T12:11:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-4.18.0-499.el8.x86_64 2/3 2023-06-27T12:11:32.851 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-4.18.0-499.el8.x86_64 2/3 2023-06-27T12:11:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:11:54.753 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:11:54.775 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:11:55.112 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:11:55.112 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-4.18.0-499.el8.x86_64 1/3 2023-06-27T12:11:55.113 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-core-4.18.0-499.el8.x86_64 2/3 2023-06-27T12:11:55.757 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-4.18.0-499.el8.x86_64 3/3 2023-06-27T12:11:55.758 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:11:55.758 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:11:55.758 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-4.18.0-499.el8.x86_64 kernel-core-4.18.0-499.el8.x86_64 2023-06-27T12:11:55.758 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-4.18.0-499.el8.x86_64 2023-06-27T12:11:55.758 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:11:55.758 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:11:55.897 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2023-06-27T12:11:56.119 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-4.18.0-499.el8.x86_64 2023-06-27T12:11:56.119 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-499.el8.x86_64 2023-06-27T12:11:56.120 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-499.el8.x86_64 2023-06-27T12:11:56.120 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-27T12:11:56.120 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-27T12:11:56.120 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-06-27T12:11:56.120 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2023-06-27T12:11:56.121 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2023-06-27T12:11:56.325 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-4.18.0-499.el8.x86_64 2023-06-27T12:11:56.326 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-499.el8.x86_64 2023-06-27T12:11:56.326 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2023-06-27T12:11:56.359 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2023-06-27T12:11:56.360 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2023-06-27T12:11:56.360 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.02 2023-06-27T12:11:56.360 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 129.el8 2023-06-27T12:11:56.360 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2023-06-27T12:11:56.360 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-06-27T12:11:56.361 INFO:teuthology.orchestra.run.smithi138.stdout:Group : System Environment/Base 2023-06-27T12:11:56.361 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 9265586 2023-06-27T12:11:56.361 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2023-06-27T12:11:56.361 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-06-27T12:11:56.361 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-06-27T12:11:56.361 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-06-27T12:11:56.361 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : kbuilder.bsys.centos.org 2023-06-27T12:11:56.362 INFO:teuthology.orchestra.run.smithi138.stdout:Relocations : (not relocatable) 2023-06-27T12:11:56.362 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : CentOS BuildSystem 2023-06-27T12:11:56.362 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2023-06-27T12:11:56.362 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-27T12:11:56.362 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2023-06-27T12:11:56.362 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2023-06-27T12:11:56.362 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:11:56.362 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-27T12:11:56.363 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-27T12:11:56.363 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-27T12:11:56.363 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2023-06-27T12:11:56.363 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:11:56.363 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2023-06-27T12:11:56.364 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-27T12:11:56.364 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 4.18.0-499.el8.x86_64 2023-06-27T12:11:56.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-27T12:11:58.152 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2023-06-27T12:12:00.101 INFO:teuthology.orchestra.run.smithi138.stderr:done 2023-06-27T12:12:00.102 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2023-06-27T12:12:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.YeXaT8LIHB 2023-06-27T12:12:00.131 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YeXaT8LIHB 2023-06-27T12:12:00.205 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.YeXaT8LIHB 2023-06-27T12:12:00.356 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.YeXaT8LIHB is 6KB 2023-06-27T12:12:00.377 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.YeXaT8LIHB 2023-06-27T12:12:00.394 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2023-06-27T12:12:00.464 INFO:teuthology.orchestra.run.smithi138.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-06-27T12:12:00.464 INFO:teuthology.orchestra.run.smithi138.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-06-27T12:12:00.464 INFO:teuthology.orchestra.run.smithi138.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-06-27T12:12:00.464 INFO:teuthology.orchestra.run.smithi138.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-499.el8.x86_64.conf 2023-06-27T12:12:00.465 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-499.el8.x86_64 2023-06-27T12:12:00.605 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2023-06-27T12:12:30.608 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-06-27T12:12:30.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-06-27T12:12:30.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-27T12:12:49.136 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-06-27T12:13:19.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-06-27T12:13:19.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-27T12:13:37.181 DEBUG:teuthology.orchestra.run.smithi138:> true 2023-06-27T12:13:37.467 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-06-27T12:13:37.468 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-499.el8.x86_64"... 2023-06-27T12:13:37.468 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2023-06-27T12:13:37.487 INFO:teuthology.orchestra.run.smithi138.stdout:4.18.0-499.el8.x86_64 2023-06-27T12:13:37.488 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-499.el8.x86_64 vs 4.18.0-499.el8.x86_64 2023-06-27T12:13:37.488 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-27T12:13:37.488 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-06-27T12:13:38.489 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-06-27T12:13:38.490 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-27T12:13:38.562 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2023-06-27T12:13:38.577 DEBUG:teuthology.parallel:result is None 2023-06-27T12:13:38.578 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-27T12:13:38.589 INFO:teuthology.task.internal:Creating test directory... 2023-06-27T12:13:38.589 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-27T12:13:38.591 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-27T12:13:38.615 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-27T12:13:38.624 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-27T12:13:38.638 INFO:teuthology.task.internal:Creating archive directory... 2023-06-27T12:13:38.638 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-27T12:13:38.650 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-27T12:13:38.699 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-27T12:13:38.711 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-27T12:13:38.711 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-27T12:13:38.732 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-27T12:13:38.768 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-27T12:13:38.779 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-27T12:13:38.785 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-27T12:13:38.795 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-27T12:13:38.797 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-27T12:13:38.808 INFO:teuthology.task.internal:Configuring sudo... 2023-06-27T12:13:38.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-27T12:13:38.812 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-27T12:13:38.852 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-27T12:13:38.865 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-27T12:13:38.866 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-27T12:13:38.880 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-27T12:13:38.912 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-27T12:13:38.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-27T12:13:39.037 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-27T12:13:39.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-27T12:13:39.164 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:13:39.165 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-27T12:13:39.233 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-27T12:13:39.285 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-27T12:13:39.334 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-27T12:13:39.426 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-27T12:13:39.464 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:13:39.465 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-27T12:13:39.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2023-06-27T12:13:39.535 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2023-06-27T12:13:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-27T12:13:39.654 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-27T12:13:40.089 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-27T12:13:40.097 INFO:teuthology.task.internal:Starting timer... 2023-06-27T12:13:40.097 INFO:teuthology.run_tasks:Running task pcp... 2023-06-27T12:13:40.108 INFO:teuthology.run_tasks:Running task selinux... 2023-06-27T12:13:40.128 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-27T12:13:40.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2023-06-27T12:13:40.209 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2023-06-27T12:13:40.211 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2023-06-27T12:13:40.281 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2023-06-27T12:13:40.283 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-27T12:13:40.283 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2023-06-27T12:13:40.311 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2023-06-27T12:13:40.312 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2023-06-27T12:13:40.334 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2023-06-27T12:13:40.335 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive'} 2023-06-27T12:13:40.335 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-27T12:13:40.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-27T12:13:40.379 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-27T12:13:40.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-27T12:13:40.407 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-27T12:13:40.407 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2023-06-27T12:13:40.446 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2023-06-27T12:13:40.474 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-27T12:13:40.486 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-06-27T12:13:40.486 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-27T12:13:40.507 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-27T12:13:40.508 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi138.front.sepia.ceph.com,smithi159.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-27T12:24:07.610 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com')] 2023-06-27T12:24:07.611 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-06-27T12:24:07.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-27T12:24:07.681 DEBUG:teuthology.orchestra.run.smithi138:> true 2023-06-27T12:24:07.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-06-27T12:24:07.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-27T12:24:07.819 DEBUG:teuthology.orchestra.run.smithi159:> true 2023-06-27T12:24:07.891 INFO:teuthology.run_tasks:Running task clock... 2023-06-27T12:24:07.903 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-27T12:24:07.903 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-27T12:24:07.904 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-27T12:24:07.906 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-27T12:24:07.906 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-27T12:24:07.938 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-27T12:24:07.953 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-27T12:24:07.961 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-27T12:24:07.975 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-27T12:24:08.028 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2023-06-27T12:24:08.040 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2023-06-27T12:24:08.053 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-27T12:24:08.063 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2023-06-27T12:24:08.067 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-27T12:24:08.075 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2023-06-27T12:24:08.088 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-27T12:24:08.102 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-27T12:24:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:bash: ntpq: command not found 2023-06-27T12:24:08.135 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-27T12:24:08.135 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2023-06-27T12:24:08.135 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-27T12:24:08.135 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-27T12:24:08.136 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-27T12:24:08.136 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-27T12:24:08.164 INFO:teuthology.orchestra.run.smithi159.stderr:bash: ntpq: command not found 2023-06-27T12:24:08.167 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-27T12:24:08.167 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2023-06-27T12:24:08.167 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-27T12:24:08.168 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-27T12:24:08.168 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-27T12:24:08.168 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-27T12:24:08.169 INFO:teuthology.run_tasks:Running task install... 2023-06-27T12:24:08.178 DEBUG:teuthology.task.install:project ceph 2023-06-27T12:24:08.178 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097'}} 2023-06-27T12:24:08.179 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097'} 2023-06-27T12:24:08.179 INFO:teuthology.task.install:Using flavor: crimson 2023-06-27T12:24:08.185 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-27T12:24:08.185 INFO:teuthology.task.install:extra packages: [] 2023-06-27T12:24:08.186 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097', 'tag': None, 'wait_for_package': False} 2023-06-27T12:24:08.186 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=ba3fff7216d65a7042e2ef64fe924764dc57a097 2023-06-27T12:24:08.189 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097', 'tag': None, 'wait_for_package': False} 2023-06-27T12:24:08.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=ba3fff7216d65a7042e2ef64fe924764dc57a097 2023-06-27T12:24:08.359 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-61651/ba3fff7216d65a7042e2ef64fe924764dc57a097/centos/8/flavors/crimson/ 2023-06-27T12:24:08.360 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4614.gba3fff72 2023-06-27T12:24:08.391 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-61651/ba3fff7216d65a7042e2ef64fe924764dc57a097/centos/8/flavors/crimson/ 2023-06-27T12:24:08.391 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4614.gba3fff72 2023-06-27T12:24:08.471 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-61651/ba3fff7216d65a7042e2ef64fe924764dc57a097/centos/8/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-61651/ba3fff7216d65a7042e2ef64fe924764dc57a097/centos/8/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-61651/ba3fff7216d65a7042e2ef64fe924764dc57a097/centos/8/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-27T12:24:08.472 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:24:08.472 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-27T12:24:08.502 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-06-27T12:24:08.502 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba3fff7216d65a7042e2ef64fe924764dc57a097/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-27T12:24:08.572 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-27T12:24:08.594 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-61651/ba3fff7216d65a7042e2ef64fe924764dc57a097/centos/8/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-61651/ba3fff7216d65a7042e2ef64fe924764dc57a097/centos/8/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-61651/ba3fff7216d65a7042e2ef64fe924764dc57a097/centos/8/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-27T12:24:08.595 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:24:08.595 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-27T12:24:08.623 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-06-27T12:24:08.624 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba3fff7216d65a7042e2ef64fe924764dc57a097/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-27T12:24:08.651 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-27T12:24:08.691 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-27T12:24:08.718 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2023-06-27T12:24:08.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2023-06-27T12:24:08.768 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-27T12:24:08.835 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2023-06-27T12:24:08.836 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2023-06-27T12:24:08.994 INFO:teuthology.orchestra.run.smithi138.stdout:73 files removed 2023-06-27T12:24:09.022 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw 2023-06-27T12:24:09.119 INFO:teuthology.orchestra.run.smithi159.stdout:73 files removed 2023-06-27T12:24:09.146 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw 2023-06-27T12:24:09.790 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 231 kB/s | 84 kB 00:00 2023-06-27T12:24:09.883 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 254 kB/s | 84 kB 00:00 2023-06-27T12:24:10.099 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 35 kB/s | 9.4 kB 00:00 2023-06-27T12:24:10.217 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 32 kB/s | 9.4 kB 00:00 2023-06-27T12:24:10.609 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 3.3 kB/s | 1.6 kB 00:00 2023-06-27T12:24:10.693 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 3.5 kB/s | 1.6 kB 00:00 2023-06-27T12:24:15.958 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - AppStream 5.7 MB/s | 30 MB 00:05 2023-06-27T12:24:23.267 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - BaseOS 23 MB/s | 40 MB 00:01 2023-06-27T12:24:29.626 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - Extras 47 kB/s | 18 kB 00:00 2023-06-27T12:24:30.485 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - Extras common packages 9.5 kB/s | 6.6 kB 00:00 2023-06-27T12:24:31.129 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 8 - PowerTools 12 MB/s | 6.1 MB 00:00 2023-06-27T12:24:34.980 INFO:teuthology.orchestra.run.smithi138.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-27T12:24:47.730 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 1.2 MB/s | 16 MB 00:12 2023-06-27T12:24:51.546 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 182 kB/s | 24 kB 00:00 2023-06-27T12:24:55.542 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:24:55.545 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:24:55.545 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:24:55.545 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:24:55.545 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:24:55.546 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 15 M 2023-06-27T12:24:55.546 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2023-06-27T12:24:55.546 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 3.3 M 2023-06-27T12:24:55.546 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 3.6 M 2023-06-27T12:24:55.546 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-06-27T12:24:55.546 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 6.8 M 2023-06-27T12:24:55.546 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 23 M 2023-06-27T12:24:55.547 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 24 k 2023-06-27T12:24:55.547 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-27T12:24:55.547 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 747 k 2023-06-27T12:24:55.547 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-27T12:24:55.547 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-06-27T12:24:55.547 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 461 k 2023-06-27T12:24:55.547 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-06-27T12:24:55.548 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 7.4 M 2023-06-27T12:24:55.548 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-27T12:24:55.548 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-06-27T12:24:55.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 45 k 2023-06-27T12:24:55.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 119 k 2023-06-27T12:24:55.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 197 k 2023-06-27T12:24:55.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-06-27T12:24:55.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 371 k 2023-06-27T12:24:55.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 365 k 2023-06-27T12:24:55.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 107 k 2023-06-27T12:24:55.549 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-27T12:24:55.549 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:24:55.549 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:24:55.549 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:24:55.550 INFO:teuthology.orchestra.run.smithi138.stdout:Install 21 Packages 2023-06-27T12:24:55.550 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2023-06-27T12:24:55.550 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:24:55.550 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 64 M 2023-06-27T12:24:55.551 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:24:59.189 INFO:teuthology.orchestra.run.smithi138.stdout:(1/23): ceph-base-18.0.0-4614.gba3fff72.el8.x86 1.9 MB/s | 6.8 MB 00:03 2023-06-27T12:24:59.256 INFO:teuthology.orchestra.run.smithi138.stdout:(2/23): ceph-selinux-18.0.0-4614.gba3fff72.el8. 366 kB/s | 24 kB 00:00 2023-06-27T12:24:59.507 INFO:teuthology.orchestra.run.smithi138.stdout:(3/23): libcephfs2-18.0.0-4614.gba3fff72.el8.x8 2.9 MB/s | 747 kB 00:00 2023-06-27T12:24:59.673 INFO:teuthology.orchestra.run.smithi138.stdout:(4/23): libradosstriper1-18.0.0-4614.gba3fff72. 2.7 MB/s | 461 kB 00:00 2023-06-27T12:25:01.932 INFO:teuthology.orchestra.run.smithi138.stdout:(5/23): librgw2-18.0.0-4614.gba3fff72.el8.x86_6 3.3 MB/s | 7.4 MB 00:02 2023-06-27T12:25:01.991 INFO:teuthology.orchestra.run.smithi138.stdout:(6/23): python3-ceph-argparse-18.0.0-4614.gba3f 769 kB/s | 45 kB 00:00 2023-06-27T12:25:02.066 INFO:teuthology.orchestra.run.smithi138.stdout:(7/23): python3-ceph-common-18.0.0-4614.gba3fff 1.5 MB/s | 119 kB 00:00 2023-06-27T12:25:02.183 INFO:teuthology.orchestra.run.smithi138.stdout:(8/23): python3-cephfs-18.0.0-4614.gba3fff72.el 1.7 MB/s | 197 kB 00:00 2023-06-27T12:25:02.342 INFO:teuthology.orchestra.run.smithi138.stdout:(9/23): python3-rados-18.0.0-4614.gba3fff72.el8 2.3 MB/s | 371 kB 00:00 2023-06-27T12:25:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:(10/23): python3-rbd-18.0.0-4614.gba3fff72.el8. 2.9 MB/s | 365 kB 00:00 2023-06-27T12:25:02.542 INFO:teuthology.orchestra.run.smithi138.stdout:(11/23): python3-rgw-18.0.0-4614.gba3fff72.el8. 1.4 MB/s | 107 kB 00:00 2023-06-27T12:25:02.760 INFO:teuthology.orchestra.run.smithi138.stdout:(12/23): librdkafka-0.11.4-3.el8.x86_64.rpm 1.6 MB/s | 354 kB 00:00 2023-06-27T12:25:02.843 INFO:teuthology.orchestra.run.smithi138.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.1 MB/s | 259 kB 00:00 2023-06-27T12:25:02.885 INFO:teuthology.orchestra.run.smithi138.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 1.0 MB/s | 44 kB 00:00 2023-06-27T12:25:03.027 INFO:teuthology.orchestra.run.smithi138.stdout:(15/23): librabbitmq-0.9.0-4.el8.x86_64.rpm 331 kB/s | 47 kB 00:00 2023-06-27T12:25:03.394 INFO:teuthology.orchestra.run.smithi138.stdout:(16/23): ceph-radosgw-18.0.0-4614.gba3fff72.el8 1.9 MB/s | 15 MB 00:07 2023-06-27T12:25:05.936 INFO:teuthology.orchestra.run.smithi138.stdout:(17/23): ceph-common-18.0.0-4614.gba3fff72.el8. 2.3 MB/s | 23 MB 00:10 2023-06-27T12:27:12.225 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-06-27T12:27:12.226 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-06-27T12:27:12.592 INFO:teuthology.orchestra.run.smithi138.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 466 B/s | 59 kB 02:09 2023-06-27T12:27:12.701 INFO:teuthology.orchestra.run.smithi138.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 2.4 kB/s | 306 kB 02:09 2023-06-27T12:27:14.951 INFO:teuthology.orchestra.run.smithi138.stdout:(20/23): librados2-18.0.0-4614.gba3fff72.el8.x8 1.5 MB/s | 3.3 MB 00:02 2023-06-27T12:27:15.643 INFO:teuthology.orchestra.run.smithi138.stdout:(21/23): librbd1-18.0.0-4614.gba3fff72.el8.x86_ 5.1 MB/s | 3.6 MB 00:00 2023-06-27T12:27:16.083 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-06-27T12:27:16.335 INFO:teuthology.orchestra.run.smithi138.stdout:(22/23): libunwind-1.3.1-3.el8.x86_64.rpm 589 B/s | 75 kB 02:10 2023-06-27T12:28:38.213 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 8 - AppStream 117 kB/s | 30 MB 04:27 2023-06-27T12:28:44.821 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 8 - BaseOS 36 MB/s | 40 MB 00:01 2023-06-27T12:28:51.346 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 8 - Extras 25 kB/s | 18 kB 00:00 2023-06-27T12:28:52.138 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 8 - Extras common packages 11 kB/s | 6.6 kB 00:00 2023-06-27T12:28:52.856 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 8 - PowerTools 11 MB/s | 6.1 MB 00:00 2023-06-27T12:28:56.483 INFO:teuthology.orchestra.run.smithi159.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-27T12:28:57.683 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-06-27T12:29:01.475 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 213 kB/s | 24 kB 00:00 2023-06-27T12:29:05.434 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:29:05.437 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:29:05.437 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:29:05.437 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:29:05.437 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:29:05.437 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 15 M 2023-06-27T12:29:05.438 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2023-06-27T12:29:05.438 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 3.3 M 2023-06-27T12:29:05.438 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 3.6 M 2023-06-27T12:29:05.438 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-06-27T12:29:05.438 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 6.8 M 2023-06-27T12:29:05.438 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 23 M 2023-06-27T12:29:05.438 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 24 k 2023-06-27T12:29:05.439 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-27T12:29:05.439 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 747 k 2023-06-27T12:29:05.439 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-27T12:29:05.439 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-06-27T12:29:05.439 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 461 k 2023-06-27T12:29:05.439 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-06-27T12:29:05.439 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 7.4 M 2023-06-27T12:29:05.439 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-27T12:29:05.440 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-06-27T12:29:05.440 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 45 k 2023-06-27T12:29:05.440 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 119 k 2023-06-27T12:29:05.440 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 197 k 2023-06-27T12:29:05.440 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-06-27T12:29:05.440 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 371 k 2023-06-27T12:29:05.440 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 365 k 2023-06-27T12:29:05.441 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 107 k 2023-06-27T12:29:05.441 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-27T12:29:05.441 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:29:05.441 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:29:05.441 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:29:05.441 INFO:teuthology.orchestra.run.smithi159.stdout:Install 21 Packages 2023-06-27T12:29:05.441 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 2 Packages 2023-06-27T12:29:05.441 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:29:05.442 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 64 M 2023-06-27T12:29:05.442 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:29:07.992 INFO:teuthology.orchestra.run.smithi159.stdout:(1/23): ceph-base-18.0.0-4614.gba3fff72.el8.x86 6.0 MB/s | 6.8 MB 00:01 2023-06-27T12:29:08.068 INFO:teuthology.orchestra.run.smithi159.stdout:(2/23): ceph-selinux-18.0.0-4614.gba3fff72.el8. 324 kB/s | 24 kB 00:00 2023-06-27T12:29:08.226 INFO:teuthology.orchestra.run.smithi159.stdout:(3/23): libcephfs2-18.0.0-4614.gba3fff72.el8.x8 4.6 MB/s | 747 kB 00:00 2023-06-27T12:29:08.335 INFO:teuthology.orchestra.run.smithi159.stdout:(4/23): libradosstriper1-18.0.0-4614.gba3fff72. 4.2 MB/s | 461 kB 00:00 2023-06-27T12:29:08.710 INFO:teuthology.orchestra.run.smithi159.stdout:(5/23): ceph-radosgw-18.0.0-4614.gba3fff72.el8. 8.0 MB/s | 15 MB 00:01 2023-06-27T12:29:08.802 INFO:teuthology.orchestra.run.smithi159.stdout:(6/23): python3-ceph-argparse-18.0.0-4614.gba3f 489 kB/s | 45 kB 00:00 2023-06-27T12:29:08.927 INFO:teuthology.orchestra.run.smithi159.stdout:(7/23): python3-ceph-common-18.0.0-4614.gba3fff 951 kB/s | 119 kB 00:00 2023-06-27T12:29:09.044 INFO:teuthology.orchestra.run.smithi159.stdout:(8/23): python3-cephfs-18.0.0-4614.gba3fff72.el 1.7 MB/s | 197 kB 00:00 2023-06-27T12:29:09.144 INFO:teuthology.orchestra.run.smithi159.stdout:(9/23): librgw2-18.0.0-4614.gba3fff72.el8.x86_6 9.1 MB/s | 7.4 MB 00:00 2023-06-27T12:29:09.170 INFO:teuthology.orchestra.run.smithi159.stdout:(10/23): python3-rados-18.0.0-4614.gba3fff72.el 2.9 MB/s | 371 kB 00:00 2023-06-27T12:29:09.395 INFO:teuthology.orchestra.run.smithi159.stdout:(11/23): ceph-common-18.0.0-4614.gba3fff72.el8. 9.3 MB/s | 23 MB 00:02 2023-06-27T12:29:09.420 INFO:teuthology.orchestra.run.smithi159.stdout:(12/23): python3-rgw-18.0.0-4614.gba3fff72.el8. 427 kB/s | 107 kB 00:00 2023-06-27T12:29:09.446 INFO:teuthology.orchestra.run.smithi159.stdout:(13/23): python3-rbd-18.0.0-4614.gba3fff72.el8. 1.2 MB/s | 365 kB 00:00 2023-06-27T12:29:09.579 INFO:teuthology.orchestra.run.smithi159.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 330 kB/s | 44 kB 00:00 2023-06-27T12:29:09.604 INFO:teuthology.orchestra.run.smithi159.stdout:(15/23): librdkafka-0.11.4-3.el8.x86_64.rpm 1.7 MB/s | 354 kB 00:00 2023-06-27T12:29:09.630 INFO:teuthology.orchestra.run.smithi159.stdout:(16/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2023-06-27T12:29:09.730 INFO:teuthology.orchestra.run.smithi159.stdout:(17/23): librabbitmq-0.9.0-4.el8.x86_64.rpm 312 kB/s | 47 kB 00:00 2023-06-27T12:29:09.889 INFO:teuthology.orchestra.run.smithi159.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 227 kB/s | 59 kB 00:00 2023-06-27T12:29:09.956 INFO:teuthology.orchestra.run.smithi159.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 872 kB/s | 306 kB 00:00 2023-06-27T12:29:10.031 INFO:teuthology.orchestra.run.smithi159.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 251 kB/s | 75 kB 00:00 2023-06-27T12:29:10.398 INFO:teuthology.orchestra.run.smithi159.stdout:(21/23): librados2-18.0.0-4614.gba3fff72.el8.x8 7.5 MB/s | 3.3 MB 00:00 2023-06-27T12:29:10.540 INFO:teuthology.orchestra.run.smithi159.stdout:(22/23): librbd1-18.0.0-4614.gba3fff72.el8.x86_ 7.0 MB/s | 3.6 MB 00:00 2023-06-27T12:29:10.582 INFO:teuthology.orchestra.run.smithi159.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 2.5 MB/s | 1.7 MB 00:00 2023-06-27T12:29:10.586 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:29:10.587 INFO:teuthology.orchestra.run.smithi159.stdout:Total 13 MB/s | 64 MB 00:05 2023-06-27T12:29:10.627 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:29:10.674 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:29:10.675 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:29:11.032 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:29:11.035 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:29:12.107 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:29:12.299 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2023-06-27T12:29:12.452 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-06-27T12:29:12.835 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-06-27T12:29:13.010 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 3/25 2023-06-27T12:29:13.107 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 3/25 2023-06-27T12:29:13.349 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:18.0.0-4614.gba3fff72.el8.x86_64 4/25 2023-06-27T12:29:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 5/25 2023-06-27T12:29:13.928 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 5/25 2023-06-27T12:29:14.093 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 6/25 2023-06-27T12:29:14.317 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 6/25 2023-06-27T12:29:14.498 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2023-06-27T12:29:14.723 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 8/25 2023-06-27T12:29:14.876 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-06-27T12:29:14.942 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-06-27T12:29:15.104 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:18.0.0-4614.gba3fff72.el8. 10/25 2023-06-27T12:29:15.472 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:18.0.0-4614.gba3fff72.el8.x86_64 11/25 2023-06-27T12:29:15.688 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:18.0.0-4614.gba3fff72.el8.x86_64 12/25 2023-06-27T12:29:15.818 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:18.0.0-4614.gba3fff72.el8.x86_6 13/25 2023-06-27T12:29:15.891 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4614.gba3fff72.el8.x86_6 13/25 2023-06-27T12:29:16.078 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-06-27T12:29:16.967 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-06-27T12:29:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:18.0.0-4614.gba3fff72.el8.x86_64 16/25 2023-06-27T12:29:17.193 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:18.0.0-4614.gba3fff72.el8.x86_64 16/25 2023-06-27T12:29:17.384 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:18.0.0-4614.gba3fff72.el8.x86_64 17/25 2023-06-27T12:29:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-06-27T12:29:17.726 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:18.0.0-4614.gba3fff72.el8.x8 19/25 2023-06-27T12:29:20.552 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 20/25 2023-06-27T12:29:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 20/25 2023-06-27T12:29:21.749 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 20/25 2023-06-27T12:29:21.990 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 21/25 2023-06-27T12:29:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 21/25 2023-06-27T12:29:22.242 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:18.0.0-4614.gba3fff72.el8.x86_64 22/25 2023-06-27T12:29:23.060 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] thrift-0.13.0-2.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/t/thrift-0.13.0-2.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-06-27T12:29:23.824 INFO:teuthology.orchestra.run.smithi138.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 14 kB/s | 1.7 MB 02:11 2023-06-27T12:29:23.824 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:29:23.825 INFO:teuthology.orchestra.run.smithi138.stdout:Total 246 kB/s | 64 MB 04:28 2023-06-27T12:29:23.865 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:29:23.911 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:29:23.912 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:29:24.261 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:29:24.265 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:29:25.339 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:29:25.530 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2023-06-27T12:29:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-06-27T12:29:26.067 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-06-27T12:29:26.265 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 3/25 2023-06-27T12:29:26.366 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 3/25 2023-06-27T12:29:26.630 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:18.0.0-4614.gba3fff72.el8.x86_64 4/25 2023-06-27T12:29:26.790 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 5/25 2023-06-27T12:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 5/25 2023-06-27T12:29:27.373 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 6/25 2023-06-27T12:29:27.575 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 6/25 2023-06-27T12:29:27.761 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2023-06-27T12:29:27.958 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 8/25 2023-06-27T12:29:28.123 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-06-27T12:29:28.177 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-06-27T12:29:28.334 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:18.0.0-4614.gba3fff72.el8. 10/25 2023-06-27T12:29:28.517 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:18.0.0-4614.gba3fff72.el8.x86_64 11/25 2023-06-27T12:29:28.717 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:18.0.0-4614.gba3fff72.el8.x86_64 12/25 2023-06-27T12:29:28.848 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:18.0.0-4614.gba3fff72.el8.x86_6 13/25 2023-06-27T12:29:28.917 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4614.gba3fff72.el8.x86_6 13/25 2023-06-27T12:29:29.132 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-06-27T12:29:30.045 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-06-27T12:29:30.189 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:18.0.0-4614.gba3fff72.el8.x86_64 16/25 2023-06-27T12:29:30.263 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:18.0.0-4614.gba3fff72.el8.x86_64 16/25 2023-06-27T12:29:30.447 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:18.0.0-4614.gba3fff72.el8.x86_64 17/25 2023-06-27T12:29:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-06-27T12:29:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:18.0.0-4614.gba3fff72.el8.x8 19/25 2023-06-27T12:29:33.665 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 20/25 2023-06-27T12:29:33.889 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 20/25 2023-06-27T12:29:34.702 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 20/25 2023-06-27T12:29:34.935 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 21/25 2023-06-27T12:29:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 21/25 2023-06-27T12:29:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:18.0.0-4614.gba3fff72.el8.x86_64 22/25 2023-06-27T12:29:41.415 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4614.gba3fff72.el8.x86_64 22/25 2023-06-27T12:29:41.416 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2023-06-27T12:29:41.416 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2023-06-27T12:29:41.416 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2023-06-27T12:29:41.416 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /run 2023-06-27T12:29:41.416 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2023-06-27T12:29:41.416 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2023-06-27T12:29:41.416 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2023-06-27T12:29:41.417 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2023-06-27T12:29:41.417 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2023-06-27T12:29:41.417 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:29:43.003 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:18.0.0-4614.gba3fff72.el8.x86_64 23/25 2023-06-27T12:29:43.024 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4614.gba3fff72.el8.x86_64 23/25 2023-06-27T12:29:43.025 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-06-27T12:29:43.224 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-06-27T12:29:43.225 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-06-27T12:29:43.924 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-06-27T12:29:43.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 1/25 2023-06-27T12:29:43.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 2/25 2023-06-27T12:29:43.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:18.0.0-4614.gba3fff72.el8.x86_64 3/25 2023-06-27T12:29:43.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:18.0.0-4614.gba3fff72.el8.x86_64 4/25 2023-06-27T12:29:43.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 5/25 2023-06-27T12:29:43.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:18.0.0-4614.gba3fff72.el8.x86_6 6/25 2023-06-27T12:29:43.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:18.0.0-4614.gba3fff72.el8.x86_64 7/25 2023-06-27T12:29:43.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4614.gba3fff72.el8. 8/25 2023-06-27T12:29:43.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:18.0.0-4614.gba3fff72.el8.x8 9/25 2023-06-27T12:29:43.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:18.0.0-4614.gba3fff72.el8.x86_64 10/25 2023-06-27T12:29:43.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:18.0.0-4614.gba3fff72.el8.x86_64 11/25 2023-06-27T12:29:43.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:18.0.0-4614.gba3fff72.el8.x86_64 12/25 2023-06-27T12:29:43.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:18.0.0-4614.gba3fff72.el8.x86_64 13/25 2023-06-27T12:29:43.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-06-27T12:29:43.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-06-27T12:29:43.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-06-27T12:29:43.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 17/25 2023-06-27T12:29:43.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-06-27T12:29:43.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-06-27T12:29:43.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-06-27T12:29:43.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-06-27T12:29:43.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 22/25 2023-06-27T12:29:43.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-06-27T12:29:43.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 24/25 2023-06-27T12:29:44.891 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-06-27T12:29:44.892 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:29:44.892 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2023-06-27T12:29:44.892 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.892 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.892 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:29:44.893 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.893 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.893 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.893 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.893 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-27T12:29:44.893 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.893 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-27T12:29:44.894 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-06-27T12:29:44.894 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.894 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-27T12:29:44.894 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.894 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-27T12:29:44.894 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-27T12:29:44.894 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.894 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.895 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.895 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-27T12:29:44.895 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.895 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.895 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:44.895 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-27T12:29:44.895 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:29:44.895 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:29:45.147 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-test 2023-06-27T12:29:45.975 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:44 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:29:47.286 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:29:47.288 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:29:47.288 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:29:47.288 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:29:47.288 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:29:47.288 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 150 M 2023-06-27T12:29:47.289 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-06-27T12:29:47.289 INFO:teuthology.orchestra.run.smithi159.stdout: cryptopp x86_64 8.6.0-1.el8 epel 1.3 M 2023-06-27T12:29:47.289 INFO:teuthology.orchestra.run.smithi159.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-06-27T12:29:47.289 INFO:teuthology.orchestra.run.smithi159.stdout: libasan6 x86_64 11.2.1-9.1.el8 appstream 402 k 2023-06-27T12:29:47.289 INFO:teuthology.orchestra.run.smithi159.stdout: libatomic x86_64 8.5.0-20.el8 baseos 26 k 2023-06-27T12:29:47.289 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 135 k 2023-06-27T12:29:47.289 INFO:teuthology.orchestra.run.smithi159.stdout: libubsan x86_64 8.5.0-20.el8 baseos 152 k 2023-06-27T12:29:47.289 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-06-27T12:29:47.290 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-06-27T12:29:47.290 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-06-27T12:29:47.290 INFO:teuthology.orchestra.run.smithi159.stdout: yaml-cpp x86_64 0.6.2-3.el8 epel 171 k 2023-06-27T12:29:47.290 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:29:47.290 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:29:47.290 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:29:47.290 INFO:teuthology.orchestra.run.smithi159.stdout:Install 11 Packages 2023-06-27T12:29:47.291 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:29:47.291 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 153 M 2023-06-27T12:29:47.291 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 1.7 G 2023-06-27T12:29:47.291 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:29:47.862 INFO:teuthology.orchestra.run.smithi159.stdout:(1/11): libcephsqlite-18.0.0-4614.gba3fff72.el8 708 kB/s | 135 kB 00:00 2023-06-27T12:29:54.487 INFO:teuthology.orchestra.run.smithi159.stdout:(2/11): ceph-test-18.0.0-4614.gba3fff72.el8.x86 22 MB/s | 150 MB 00:06 2023-06-27T12:29:54.716 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4614.gba3fff72.el8.x86_64 22/25 2023-06-27T12:29:54.716 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2023-06-27T12:29:54.716 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2023-06-27T12:29:54.717 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /dev 2023-06-27T12:29:54.717 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /run 2023-06-27T12:29:54.717 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2023-06-27T12:29:54.717 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2023-06-27T12:29:54.717 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2023-06-27T12:29:54.717 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2023-06-27T12:29:54.717 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /dev 2023-06-27T12:29:54.718 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:29:56.340 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:18.0.0-4614.gba3fff72.el8.x86_64 23/25 2023-06-27T12:29:56.360 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4614.gba3fff72.el8.x86_64 23/25 2023-06-27T12:29:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-06-27T12:29:56.571 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-06-27T12:29:56.571 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-06-27T12:29:57.248 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-06-27T12:29:57.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 1/25 2023-06-27T12:29:57.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 2/25 2023-06-27T12:29:57.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:18.0.0-4614.gba3fff72.el8.x86_64 3/25 2023-06-27T12:29:57.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:18.0.0-4614.gba3fff72.el8.x86_64 4/25 2023-06-27T12:29:57.249 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 5/25 2023-06-27T12:29:57.249 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:18.0.0-4614.gba3fff72.el8.x86_6 6/25 2023-06-27T12:29:57.249 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:18.0.0-4614.gba3fff72.el8.x86_64 7/25 2023-06-27T12:29:57.249 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4614.gba3fff72.el8. 8/25 2023-06-27T12:29:57.249 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:18.0.0-4614.gba3fff72.el8.x8 9/25 2023-06-27T12:29:57.249 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:18.0.0-4614.gba3fff72.el8.x86_64 10/25 2023-06-27T12:29:57.249 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:18.0.0-4614.gba3fff72.el8.x86_64 11/25 2023-06-27T12:29:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:18.0.0-4614.gba3fff72.el8.x86_64 12/25 2023-06-27T12:29:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:18.0.0-4614.gba3fff72.el8.x86_64 13/25 2023-06-27T12:29:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-06-27T12:29:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-06-27T12:29:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-06-27T12:29:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 17/25 2023-06-27T12:29:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-06-27T12:29:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-06-27T12:29:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-06-27T12:29:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-06-27T12:29:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 22/25 2023-06-27T12:29:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-06-27T12:29:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 24/25 2023-06-27T12:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-06-27T12:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2023-06-27T12:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-27T12:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-27T12:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-06-27T12:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-27T12:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-27T12:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-27T12:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-27T12:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-27T12:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:29:58.592 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-test 2023-06-27T12:29:59.416 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:05:08 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:30:00.742 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:30:00.743 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:30:00.743 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:30:00.743 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:30:00.744 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:30:00.744 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 150 M 2023-06-27T12:30:00.744 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-06-27T12:30:00.744 INFO:teuthology.orchestra.run.smithi138.stdout: cryptopp x86_64 8.6.0-1.el8 epel 1.3 M 2023-06-27T12:30:00.744 INFO:teuthology.orchestra.run.smithi138.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-06-27T12:30:00.744 INFO:teuthology.orchestra.run.smithi138.stdout: libasan6 x86_64 11.2.1-9.1.el8 appstream 402 k 2023-06-27T12:30:00.745 INFO:teuthology.orchestra.run.smithi138.stdout: libatomic x86_64 8.5.0-20.el8 baseos 26 k 2023-06-27T12:30:00.745 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 135 k 2023-06-27T12:30:00.745 INFO:teuthology.orchestra.run.smithi138.stdout: libubsan x86_64 8.5.0-20.el8 baseos 152 k 2023-06-27T12:30:00.745 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-06-27T12:30:00.745 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-06-27T12:30:00.745 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-06-27T12:30:00.745 INFO:teuthology.orchestra.run.smithi138.stdout: yaml-cpp x86_64 0.6.2-3.el8 epel 171 k 2023-06-27T12:30:00.746 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:30:00.746 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:30:00.746 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:30:00.746 INFO:teuthology.orchestra.run.smithi138.stdout:Install 11 Packages 2023-06-27T12:30:00.746 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:30:00.747 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 153 M 2023-06-27T12:30:00.747 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 1.7 G 2023-06-27T12:30:00.747 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:30:01.416 INFO:teuthology.orchestra.run.smithi138.stdout:(1/11): libcephsqlite-18.0.0-4614.gba3fff72.el8 456 kB/s | 135 kB 00:00 2023-06-27T12:30:01.500 INFO:teuthology.orchestra.run.smithi138.stdout:(2/11): jq-1.6-7.el8.x86_64.rpm 534 kB/s | 202 kB 00:00 2023-06-27T12:30:01.842 INFO:teuthology.orchestra.run.smithi138.stdout:(3/11): oniguruma-6.8.2-2.el8.x86_64.rpm 548 kB/s | 187 kB 00:00 2023-06-27T12:30:01.867 INFO:teuthology.orchestra.run.smithi138.stdout:(4/11): libasan6-11.2.1-9.1.el8.x86_64.rpm 891 kB/s | 402 kB 00:00 2023-06-27T12:30:02.026 INFO:teuthology.orchestra.run.smithi138.stdout:(5/11): xmlstarlet-1.6.1-20.el8.x86_64.rpm 446 kB/s | 71 kB 00:00 2023-06-27T12:30:02.134 INFO:teuthology.orchestra.run.smithi138.stdout:(6/11): libatomic-8.5.0-20.el8.x86_64.rpm 236 kB/s | 26 kB 00:00 2023-06-27T12:30:02.252 INFO:teuthology.orchestra.run.smithi138.stdout:(7/11): libubsan-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 152 kB 00:00 2023-06-27T12:30:02.560 INFO:teuthology.orchestra.run.smithi138.stdout:(8/11): cryptopp-8.6.0-1.el8.x86_64.rpm 4.1 MB/s | 1.3 MB 00:00 2023-06-27T12:30:02.619 INFO:teuthology.orchestra.run.smithi138.stdout:(9/11): yaml-cpp-0.6.2-3.el8.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2023-06-27T12:30:02.644 INFO:teuthology.orchestra.run.smithi138.stdout:(10/11): socat-1.7.4.1-1.el8.x86_64.rpm 403 kB/s | 323 kB 00:00 2023-06-27T12:30:08.161 INFO:teuthology.orchestra.run.smithi138.stdout:(11/11): ceph-test-18.0.0-4614.gba3fff72.el8.x8 21 MB/s | 150 MB 00:07 2023-06-27T12:30:08.162 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:30:08.163 INFO:teuthology.orchestra.run.smithi138.stdout:Total 21 MB/s | 153 MB 00:07 2023-06-27T12:30:08.236 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:30:08.259 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:30:08.259 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:30:08.970 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:30:08.973 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:30:09.975 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:30:10.255 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : yaml-cpp-0.6.2-3.el8.x86_64 1/11 2023-06-27T12:30:10.420 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cryptopp-8.6.0-1.el8.x86_64 2/11 2023-06-27T12:30:10.541 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libubsan-8.5.0-20.el8.x86_64 3/11 2023-06-27T12:30:10.592 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libubsan-8.5.0-20.el8.x86_64 3/11 2023-06-27T12:30:10.691 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libatomic-8.5.0-20.el8.x86_64 4/11 2023-06-27T12:30:10.727 INFO:teuthology.orchestra.run.smithi159.stdout:(3/11): oniguruma-6.8.2-2.el8.x86_64.rpm 12 kB/s | 187 kB 00:16 2023-06-27T12:30:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libatomic-8.5.0-20.el8.x86_64 4/11 2023-06-27T12:30:10.961 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 5/11 2023-06-27T12:30:11.136 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 6/11 2023-06-27T12:30:11.160 INFO:teuthology.orchestra.run.smithi159.stdout:(4/11): socat-1.7.4.1-1.el8.x86_64.rpm 746 kB/s | 323 kB 00:00 2023-06-27T12:30:11.250 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 7/11 2023-06-27T12:30:11.277 INFO:teuthology.orchestra.run.smithi159.stdout:(5/11): xmlstarlet-1.6.1-20.el8.x86_64.rpm 605 kB/s | 71 kB 00:00 2023-06-27T12:30:11.327 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 7/11 2023-06-27T12:30:11.377 INFO:teuthology.orchestra.run.smithi159.stdout:(6/11): libatomic-8.5.0-20.el8.x86_64.rpm 256 kB/s | 26 kB 00:00 2023-06-27T12:30:11.492 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : jq-1.6-7.el8.x86_64 8/11 2023-06-27T12:30:11.494 INFO:teuthology.orchestra.run.smithi159.stdout:(7/11): libubsan-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 152 kB 00:00 2023-06-27T12:30:11.624 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-06-27T12:30:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-06-27T12:30:11.824 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:18.0.0-4614.gba3fff72.el8.x86_64 10/11 2023-06-27T12:30:13.394 INFO:teuthology.orchestra.run.smithi159.stdout:(8/11): cryptopp-8.6.0-1.el8.x86_64.rpm 687 kB/s | 1.3 MB 00:01 2023-06-27T12:30:13.603 INFO:teuthology.orchestra.run.smithi159.stdout:(9/11): yaml-cpp-0.6.2-3.el8.x86_64.rpm 819 kB/s | 171 kB 00:00 2023-06-27T12:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4614.gba3fff72.el8.x86_64 10/11 2023-06-27T12:30:35.269 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:18.0.0-4614.gba3fff72.el8.x86_64 11/11 2023-06-27T12:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-test-2:18.0.0-4614.gba3fff72.el8.x86_64 11/11 2023-06-27T12:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:18.0.0-4614.gba3fff72.el8.x86_64 1/11 2023-06-27T12:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:18.0.0-4614.gba3fff72.el8.x86_64 2/11 2023-06-27T12:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : jq-1.6-7.el8.x86_64 3/11 2023-06-27T12:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libasan6-11.2.1-9.1.el8.x86_64 4/11 2023-06-27T12:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/11 2023-06-27T12:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 6/11 2023-06-27T12:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 7/11 2023-06-27T12:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libatomic-8.5.0-20.el8.x86_64 8/11 2023-06-27T12:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libubsan-8.5.0-20.el8.x86_64 9/11 2023-06-27T12:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cryptopp-8.6.0-1.el8.x86_64 10/11 2023-06-27T12:30:36.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : yaml-cpp-0.6.2-3.el8.x86_64 11/11 2023-06-27T12:30:36.231 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:30:36.231 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:30:36.231 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:30:36.232 INFO:teuthology.orchestra.run.smithi138.stdout: cryptopp-8.6.0-1.el8.x86_64 2023-06-27T12:30:36.232 INFO:teuthology.orchestra.run.smithi138.stdout: jq-1.6-7.el8.x86_64 2023-06-27T12:30:36.232 INFO:teuthology.orchestra.run.smithi138.stdout: libasan6-11.2.1-9.1.el8.x86_64 2023-06-27T12:30:36.232 INFO:teuthology.orchestra.run.smithi138.stdout: libatomic-8.5.0-20.el8.x86_64 2023-06-27T12:30:36.232 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:30:36.232 INFO:teuthology.orchestra.run.smithi138.stdout: libubsan-8.5.0-20.el8.x86_64 2023-06-27T12:30:36.232 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-27T12:30:36.233 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-27T12:30:36.233 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-27T12:30:36.233 INFO:teuthology.orchestra.run.smithi138.stdout: yaml-cpp-0.6.2-3.el8.x86_64 2023-06-27T12:30:36.233 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:30:36.233 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:30:36.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph 2023-06-27T12:30:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:05:46 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:30:38.654 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:30:38.657 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:30:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:30:38.657 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:30:38.657 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:30:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 6.4 k 2023-06-27T12:30:38.658 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-06-27T12:30:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 2.2 M 2023-06-27T12:30:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 1.5 M 2023-06-27T12:30:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 240 k 2023-06-27T12:30:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 9.0 M 2023-06-27T12:30:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 159 M 2023-06-27T12:30:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-27T12:30:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-27T12:30:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-06-27T12:30:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-27T12:30:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-27T12:30:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-06-27T12:30:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-27T12:30:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-27T12:30:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-27T12:30:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-06-27T12:30:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-06-27T12:30:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-27T12:30:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-27T12:30:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-27T12:30:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-06-27T12:30:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-06-27T12:30:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2017.2-9.el8 appstream 54 k 2023-06-27T12:30:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-27T12:30:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-27T12:30:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-27T12:30:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-27T12:30:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-27T12:30:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-27T12:30:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-27T12:30:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-06-27T12:30:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-27T12:30:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:30:38.662 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:30:38.663 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:30:38.663 INFO:teuthology.orchestra.run.smithi138.stdout:Install 32 Packages 2023-06-27T12:30:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:30:38.663 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 175 M 2023-06-27T12:30:38.664 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 2.3 G 2023-06-27T12:30:38.664 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:30:39.085 INFO:teuthology.orchestra.run.smithi138.stdout:(1/32): ceph-18.0.0-4614.gba3fff72.el8.x86_64.r 46 kB/s | 6.4 kB 00:00 2023-06-27T12:30:39.269 INFO:teuthology.orchestra.run.smithi138.stdout:(2/32): ceph-mgr-18.0.0-4614.gba3fff72.el8.x86_ 4.6 MB/s | 1.5 MB 00:00 2023-06-27T12:30:39.311 INFO:teuthology.orchestra.run.smithi138.stdout:(3/32): ceph-mds-18.0.0-4614.gba3fff72.el8.x86_ 6.0 MB/s | 2.2 MB 00:00 2023-06-27T12:30:39.386 INFO:teuthology.orchestra.run.smithi138.stdout:(4/32): ceph-mgr-modules-core-18.0.0-4614.gba3f 3.1 MB/s | 240 kB 00:00 2023-06-27T12:30:42.012 INFO:teuthology.orchestra.run.smithi138.stdout:(5/32): ceph-mon-18.0.0-4614.gba3fff72.el8.x86_ 3.1 MB/s | 9.0 MB 00:02 2023-06-27T12:30:47.831 INFO:teuthology.orchestra.run.smithi138.stdout:(6/32): ceph-osd-18.0.0-4614.gba3fff72.el8.x86_ 19 MB/s | 159 MB 00:08 2023-06-27T12:31:43.731 INFO:teuthology.orchestra.run.smithi138.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 2.4 kB/s | 157 kB 01:04 2023-06-27T12:31:43.789 INFO:teuthology.orchestra.run.smithi138.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 931 kB/s | 54 kB 00:00 2023-06-27T12:31:44.006 INFO:teuthology.orchestra.run.smithi138.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 2.1 MB/s | 457 kB 00:00 2023-06-27T12:31:44.215 INFO:teuthology.orchestra.run.smithi138.stdout:(10/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.2 MB/s | 249 kB 00:00 2023-06-27T12:31:44.323 INFO:teuthology.orchestra.run.smithi138.stdout:(11/32): python3-cryptography-3.2.1-6.el8.x86_6 5.8 MB/s | 641 kB 00:00 2023-06-27T12:31:44.373 INFO:teuthology.orchestra.run.smithi138.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 2.1 MB/s | 109 kB 00:00 2023-06-27T12:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 158 kB/s | 44 kB 00:00 2023-06-27T12:31:44.849 INFO:teuthology.orchestra.run.smithi138.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 927 kB/s | 185 kB 00:00 2023-06-27T12:31:45.016 INFO:teuthology.orchestra.run.smithi138.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.0 MB/s | 173 kB 00:00 2023-06-27T12:31:45.350 INFO:teuthology.orchestra.run.smithi138.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.1 MB/s | 384 kB 00:00 2023-06-27T12:31:45.433 INFO:teuthology.orchestra.run.smithi138.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-06-27T12:31:45.500 INFO:teuthology.orchestra.run.smithi138.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 264 kB/s | 18 kB 00:00 2023-06-27T12:31:45.567 INFO:teuthology.orchestra.run.smithi138.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 731 kB/s | 49 kB 00:00 2023-06-27T12:31:45.634 INFO:teuthology.orchestra.run.smithi138.stdout:(20/32): python3-more-itertools-7.2.0-3.el8.noa 881 kB/s | 59 kB 00:00 2023-06-27T12:31:45.909 INFO:teuthology.orchestra.run.smithi138.stdout:(21/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.0 MB/s | 283 kB 00:00 2023-06-27T12:31:45.976 INFO:teuthology.orchestra.run.smithi138.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 234 kB/s | 16 kB 00:00 2023-06-27T12:31:46.010 INFO:teuthology.orchestra.run.smithi138.stdout:(23/32): python3-markupsafe-0.23-19.el8.x86_64. 620 B/s | 39 kB 01:03 2023-06-27T12:31:46.044 INFO:teuthology.orchestra.run.smithi138.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 279 kB/s | 19 kB 00:00 2023-06-27T12:31:46.111 INFO:teuthology.orchestra.run.smithi138.stdout:(25/32): python3-tempora-1.14.1-5.el8.noarch.rp 439 kB/s | 29 kB 00:00 2023-06-27T12:31:46.186 INFO:teuthology.orchestra.run.smithi138.stdout:(26/32): python3-trustme-0.6.0-4.el8.noarch.rpm 369 kB/s | 27 kB 00:00 2023-06-27T12:31:46.211 INFO:teuthology.orchestra.run.smithi138.stdout:(27/32): python3-singledispatch-3.4.0.3-18.el8. 119 kB/s | 24 kB 00:00 2023-06-27T12:31:46.420 INFO:teuthology.orchestra.run.smithi138.stdout:(28/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.2 MB/s | 251 kB 00:00 2023-06-27T12:31:46.462 INFO:teuthology.orchestra.run.smithi138.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 926 kB/s | 255 kB 00:00 2023-06-27T12:31:46.521 INFO:teuthology.orchestra.run.smithi138.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 852 kB/s | 85 kB 00:00 2023-06-27T12:31:46.554 INFO:teuthology.orchestra.run.smithi138.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 250 kB/s | 23 kB 00:00 2023-06-27T12:31:59.396 INFO:teuthology.orchestra.run.smithi159.stdout:(10/11): jq-1.6-7.el8.x86_64.rpm 1.5 kB/s | 202 kB 02:11 2023-06-27T12:31:59.421 INFO:teuthology.orchestra.run.smithi159.stdout:(11/11): libasan6-11.2.1-9.1.el8.x86_64.rpm 3.1 kB/s | 402 kB 02:11 2023-06-27T12:31:59.424 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:31:59.424 INFO:teuthology.orchestra.run.smithi159.stdout:Total 1.2 MB/s | 153 MB 02:12 2023-06-27T12:31:59.489 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:31:59.513 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:31:59.513 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:32:00.222 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:32:00.225 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:32:01.222 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:32:01.497 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : yaml-cpp-0.6.2-3.el8.x86_64 1/11 2023-06-27T12:32:01.688 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cryptopp-8.6.0-1.el8.x86_64 2/11 2023-06-27T12:32:01.809 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libubsan-8.5.0-20.el8.x86_64 3/11 2023-06-27T12:32:01.877 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libubsan-8.5.0-20.el8.x86_64 3/11 2023-06-27T12:32:01.976 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libatomic-8.5.0-20.el8.x86_64 4/11 2023-06-27T12:32:02.045 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libatomic-8.5.0-20.el8.x86_64 4/11 2023-06-27T12:32:02.254 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 5/11 2023-06-27T12:32:02.453 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 6/11 2023-06-27T12:32:02.576 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 7/11 2023-06-27T12:32:02.653 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 7/11 2023-06-27T12:32:02.826 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : jq-1.6-7.el8.x86_64 8/11 2023-06-27T12:32:02.934 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-06-27T12:32:02.991 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-06-27T12:32:03.142 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:18.0.0-4614.gba3fff72.el8.x86_64 10/11 2023-06-27T12:32:26.361 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4614.gba3fff72.el8.x86_64 10/11 2023-06-27T12:32:26.596 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:18.0.0-4614.gba3fff72.el8.x86_64 11/11 2023-06-27T12:32:26.885 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-test-2:18.0.0-4614.gba3fff72.el8.x86_64 11/11 2023-06-27T12:32:26.885 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:18.0.0-4614.gba3fff72.el8.x86_64 1/11 2023-06-27T12:32:26.885 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:18.0.0-4614.gba3fff72.el8.x86_64 2/11 2023-06-27T12:32:26.885 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : jq-1.6-7.el8.x86_64 3/11 2023-06-27T12:32:26.885 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libasan6-11.2.1-9.1.el8.x86_64 4/11 2023-06-27T12:32:26.885 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/11 2023-06-27T12:32:26.886 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 6/11 2023-06-27T12:32:26.886 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 7/11 2023-06-27T12:32:26.887 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libatomic-8.5.0-20.el8.x86_64 8/11 2023-06-27T12:32:26.887 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libubsan-8.5.0-20.el8.x86_64 9/11 2023-06-27T12:32:26.887 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cryptopp-8.6.0-1.el8.x86_64 10/11 2023-06-27T12:32:27.547 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : yaml-cpp-0.6.2-3.el8.x86_64 11/11 2023-06-27T12:32:27.548 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:32:27.548 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:32:27.548 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:32:27.548 INFO:teuthology.orchestra.run.smithi159.stdout: cryptopp-8.6.0-1.el8.x86_64 2023-06-27T12:32:27.548 INFO:teuthology.orchestra.run.smithi159.stdout: jq-1.6-7.el8.x86_64 2023-06-27T12:32:27.549 INFO:teuthology.orchestra.run.smithi159.stdout: libasan6-11.2.1-9.1.el8.x86_64 2023-06-27T12:32:27.549 INFO:teuthology.orchestra.run.smithi159.stdout: libatomic-8.5.0-20.el8.x86_64 2023-06-27T12:32:27.549 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:32:27.549 INFO:teuthology.orchestra.run.smithi159.stdout: libubsan-8.5.0-20.el8.x86_64 2023-06-27T12:32:27.549 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-27T12:32:27.549 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-27T12:32:27.549 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-27T12:32:27.550 INFO:teuthology.orchestra.run.smithi159.stdout: yaml-cpp-0.6.2-3.el8.x86_64 2023-06-27T12:32:27.550 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:32:27.550 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:32:27.801 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph 2023-06-27T12:32:28.636 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:03:27 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:32:29.943 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:32:29.947 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:32:29.947 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:32:29.947 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:32:29.947 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:32:29.947 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 6.4 k 2023-06-27T12:32:29.947 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-06-27T12:32:29.948 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 2.2 M 2023-06-27T12:32:29.948 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 1.5 M 2023-06-27T12:32:29.948 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 240 k 2023-06-27T12:32:29.948 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 9.0 M 2023-06-27T12:32:29.948 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 159 M 2023-06-27T12:32:29.948 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-27T12:32:29.948 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-27T12:32:29.949 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-06-27T12:32:29.949 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-27T12:32:29.949 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-27T12:32:29.949 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-06-27T12:32:29.949 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-27T12:32:29.949 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-27T12:32:29.949 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-27T12:32:29.950 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-06-27T12:32:29.950 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-06-27T12:32:29.950 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-27T12:32:29.950 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-27T12:32:29.950 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-27T12:32:29.950 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-06-27T12:32:29.950 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-06-27T12:32:29.951 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2017.2-9.el8 appstream 54 k 2023-06-27T12:32:29.951 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-27T12:32:29.951 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-27T12:32:29.951 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-27T12:32:29.951 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-27T12:32:29.951 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-27T12:32:29.951 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-27T12:32:29.951 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-27T12:32:29.952 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-06-27T12:32:29.952 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-27T12:32:29.952 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:32:29.952 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:32:29.952 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:32:29.952 INFO:teuthology.orchestra.run.smithi159.stdout:Install 32 Packages 2023-06-27T12:32:29.952 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:32:29.953 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 175 M 2023-06-27T12:32:29.953 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 2.3 G 2023-06-27T12:32:29.953 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:32:30.550 INFO:teuthology.orchestra.run.smithi159.stdout:(1/32): ceph-18.0.0-4614.gba3fff72.el8.x86_64.r 53 kB/s | 6.4 kB 00:00 2023-06-27T12:32:30.775 INFO:teuthology.orchestra.run.smithi159.stdout:(2/32): ceph-mgr-18.0.0-4614.gba3fff72.el8.x86_ 4.3 MB/s | 1.5 MB 00:00 2023-06-27T12:32:32.393 INFO:teuthology.orchestra.run.smithi159.stdout:(3/32): ceph-mds-18.0.0-4614.gba3fff72.el8.x86_ 1.1 MB/s | 2.2 MB 00:01 2023-06-27T12:32:32.843 INFO:teuthology.orchestra.run.smithi159.stdout:(4/32): ceph-mon-18.0.0-4614.gba3fff72.el8.x86_ 3.9 MB/s | 9.0 MB 00:02 2023-06-27T12:32:33.135 INFO:teuthology.orchestra.run.smithi159.stdout:(5/32): ceph-mgr-modules-core-18.0.0-4614.gba3f 324 kB/s | 240 kB 00:00 2023-06-27T12:32:33.485 INFO:teuthology.orchestra.run.smithi159.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 111 kB/s | 39 kB 00:00 2023-06-27T12:32:33.794 INFO:teuthology.orchestra.run.smithi159.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 165 kB/s | 157 kB 00:00 2023-06-27T12:32:34.086 INFO:teuthology.orchestra.run.smithi159.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 172 kB/s | 103 kB 00:00 2023-06-27T12:32:34.403 INFO:teuthology.orchestra.run.smithi159.stdout:(9/32): python3-pytz-2017.2-9.el8.noarch.rpm 89 kB/s | 54 kB 00:00 2023-06-27T12:32:34.970 INFO:teuthology.orchestra.run.smithi159.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 518 kB/s | 457 kB 00:00 2023-06-27T12:32:35.278 INFO:teuthology.orchestra.run.smithi159.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 284 kB/s | 249 kB 00:00 2023-06-27T12:32:35.596 INFO:teuthology.orchestra.run.smithi159.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 344 kB/s | 109 kB 00:00 2023-06-27T12:32:35.971 INFO:teuthology.orchestra.run.smithi159.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 116 kB/s | 44 kB 00:00 2023-06-27T12:32:36.304 INFO:teuthology.orchestra.run.smithi159.stdout:(14/32): python3-cryptography-3.2.1-6.el8.x86_6 481 kB/s | 641 kB 00:01 2023-06-27T12:32:36.563 INFO:teuthology.orchestra.run.smithi159.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 313 kB/s | 185 kB 00:00 2023-06-27T12:32:36.938 INFO:teuthology.orchestra.run.smithi159.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.0 MB/s | 384 kB 00:00 2023-06-27T12:32:37.205 INFO:teuthology.orchestra.run.smithi159.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 192 kB/s | 173 kB 00:00 2023-06-27T12:32:37.489 INFO:teuthology.orchestra.run.smithi159.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 20 kB/s | 11 kB 00:00 2023-06-27T12:32:37.798 INFO:teuthology.orchestra.run.smithi159.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 30 kB/s | 18 kB 00:00 2023-06-27T12:32:38.106 INFO:teuthology.orchestra.run.smithi159.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 79 kB/s | 49 kB 00:00 2023-06-27T12:32:38.415 INFO:teuthology.orchestra.run.smithi159.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 95 kB/s | 59 kB 00:00 2023-06-27T12:32:38.674 INFO:teuthology.orchestra.run.smithi159.stdout:(22/32): python3-pecan-1.3.2-9.el8.noarch.rpm 499 kB/s | 283 kB 00:00 2023-06-27T12:32:38.983 INFO:teuthology.orchestra.run.smithi159.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 27 kB/s | 16 kB 00:00 2023-06-27T12:32:39.299 INFO:teuthology.orchestra.run.smithi159.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 30 kB/s | 19 kB 00:00 2023-06-27T12:32:39.608 INFO:teuthology.orchestra.run.smithi159.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 38 kB/s | 24 kB 00:00 2023-06-27T12:32:39.941 INFO:teuthology.orchestra.run.smithi159.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 46 kB/s | 29 kB 00:00 2023-06-27T12:32:40.250 INFO:teuthology.orchestra.run.smithi159.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 43 kB/s | 27 kB 00:00 2023-06-27T12:32:40.551 INFO:teuthology.orchestra.run.smithi159.stdout:(28/32): python3-waitress-1.4.3-1.el8.noarch.rp 419 kB/s | 255 kB 00:00 2023-06-27T12:32:40.842 INFO:teuthology.orchestra.run.smithi159.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 424 kB/s | 251 kB 00:00 2023-06-27T12:32:41.109 INFO:teuthology.orchestra.run.smithi159.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 153 kB/s | 85 kB 00:00 2023-06-27T12:32:41.385 INFO:teuthology.orchestra.run.smithi159.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 42 kB/s | 23 kB 00:00 2023-06-27T12:32:46.134 INFO:teuthology.orchestra.run.smithi159.stdout:(32/32): ceph-osd-18.0.0-4614.gba3fff72.el8.x86 10 MB/s | 159 MB 00:15 2023-06-27T12:32:46.139 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:32:46.139 INFO:teuthology.orchestra.run.smithi159.stdout:Total 11 MB/s | 175 MB 00:16 2023-06-27T12:32:46.221 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:32:46.260 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:32:46.260 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:32:47.111 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:32:47.114 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:32:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:32:48.966 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-06-27T12:32:49.176 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-06-27T12:32:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-06-27T12:32:49.608 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2023-06-27T12:32:49.773 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-06-27T12:32:49.961 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-06-27T12:32:50.121 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-06-27T12:32:50.282 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-06-27T12:32:50.463 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-06-27T12:32:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-06-27T12:32:50.930 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-06-27T12:32:51.179 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-06-27T12:32:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2023-06-27T12:32:51.763 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 14/32 2023-06-27T12:32:51.934 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-06-27T12:32:52.142 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-06-27T12:32:52.327 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-06-27T12:32:52.618 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-06-27T12:32:52.832 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-06-27T12:32:53.042 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-06-27T12:32:53.248 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-06-27T12:32:53.503 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-06-27T12:32:53.721 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-06-27T12:32:53.936 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-06-27T12:32:54.233 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-06-27T12:32:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-06-27T12:32:54.898 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4614.gba3fff72.el8. 27/32 2023-06-27T12:32:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 28/32 2023-06-27T12:33:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 28/32 2023-06-27T12:33:23.395 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:18.0.0-4614.gba3fff72.el8.x86_64 29/32 2023-06-27T12:33:24.312 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:18.0.0-4614.gba3fff72.el8.x86_64 29/32 2023-06-27T12:33:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:18.0.0-4614.gba3fff72.el8.x86_64 30/32 2023-06-27T12:33:24.715 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:18.0.0-4614.gba3fff72.el8.x86_64 30/32 2023-06-27T12:33:24.880 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:18.0.0-4614.gba3fff72.el8.x86_64 31/32 2023-06-27T12:33:24.897 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:18.0.0-4614.gba3fff72.el8.x86_64 31/32 2023-06-27T12:33:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:18.0.0-4614.gba3fff72.el8.x86_64 32/32 2023-06-27T12:33:25.445 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-2:18.0.0-4614.gba3fff72.el8.x86_64 32/32 2023-06-27T12:33:25.445 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:18.0.0-4614.gba3fff72.el8.x86_64 1/32 2023-06-27T12:33:25.445 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:18.0.0-4614.gba3fff72.el8.x86_64 2/32 2023-06-27T12:33:25.445 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 3/32 2023-06-27T12:33:25.446 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:18.0.0-4614.gba3fff72.el8.x86_64 4/32 2023-06-27T12:33:25.446 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:18.0.0-4614.gba3fff72.el8.x86_64 5/32 2023-06-27T12:33:25.446 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4614.gba3fff72.el8. 6/32 2023-06-27T12:33:25.446 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-06-27T12:33:25.446 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-06-27T12:33:25.446 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-06-27T12:33:25.446 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-06-27T12:33:25.447 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-06-27T12:33:25.447 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2023-06-27T12:33:25.447 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 13/32 2023-06-27T12:33:25.447 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-06-27T12:33:25.447 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-06-27T12:33:25.447 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-06-27T12:33:25.447 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-06-27T12:33:25.447 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-06-27T12:33:25.448 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-06-27T12:33:25.448 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-06-27T12:33:25.448 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-06-27T12:33:25.448 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-06-27T12:33:25.448 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-06-27T12:33:25.448 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-06-27T12:33:25.448 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-06-27T12:33:25.449 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-06-27T12:33:25.449 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-06-27T12:33:25.449 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-06-27T12:33:25.450 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2023-06-27T12:33:25.450 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-06-27T12:33:25.450 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-06-27T12:33:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-06-27T12:33:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:26.477 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:33:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:33:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:33:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:33:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:33:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:33:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:33:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-27T12:33:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-27T12:33:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-06-27T12:33:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-27T12:33:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-27T12:33:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-06-27T12:33:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-27T12:33:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-27T12:33:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-27T12:33:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.0.6-14.el8.noarch 2023-06-27T12:33:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-27T12:33:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-27T12:33:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-27T12:33:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-2.6-1.el8.noarch 2023-06-27T12:33:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-27T12:33:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-27T12:33:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-27T12:33:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-27T12:33:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-27T12:33:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-27T12:33:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-27T12:33:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-27T12:33:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-27T12:33:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-27T12:33:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-27T12:33:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-27T12:33:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:26.482 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:33:26.631 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-base 2023-06-27T12:33:27.465 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:04:26 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:33:28.642 INFO:teuthology.orchestra.run.smithi159.stdout:Package ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:33:28.796 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:33:28.797 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2023-06-27T12:33:28.797 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:33:28.904 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install cephadm 2023-06-27T12:33:29.732 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:04:28 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:33:31.062 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:33:31.063 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:33:31.063 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:33:31.063 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:33:31.064 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:33:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 166 k 2023-06-27T12:33:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:31.064 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:33:31.064 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:33:31.064 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2023-06-27T12:33:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:31.064 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 166 k 2023-06-27T12:33:31.065 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 654 k 2023-06-27T12:33:31.065 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:33:31.588 INFO:teuthology.orchestra.run.smithi159.stdout:cephadm-18.0.0-4614.gba3fff72.el8.noarch.rpm 318 kB/s | 166 kB 00:00 2023-06-27T12:33:31.588 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:33:31.589 INFO:teuthology.orchestra.run.smithi159.stdout:Total 317 kB/s | 166 kB 00:00 2023-06-27T12:33:31.589 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:33:31.598 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:33:31.598 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:33:31.601 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:33:31.605 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:33:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:33:38.590 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:33:40.430 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:33:40.674 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:33:41.953 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:33:41.953 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:33:41.954 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:33:41.954 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:33:43.528 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-immutable-object-cache 2023-06-27T12:33:44.364 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:04:43 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:33:45.713 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:33:45.714 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:33:45.714 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repo Size 2023-06-27T12:33:45.714 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:33:45.714 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:33:45.715 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 156 k 2023-06-27T12:33:45.715 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:45.715 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:33:45.715 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:33:45.715 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2023-06-27T12:33:45.715 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:45.715 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 156 k 2023-06-27T12:33:45.716 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 540 k 2023-06-27T12:33:45.716 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:33:46.009 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-immutable-object-cache-18.0.0-4614.gba3fff 533 kB/s | 156 kB 00:00 2023-06-27T12:33:46.009 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:33:46.010 INFO:teuthology.orchestra.run.smithi159.stdout:Total 531 kB/s | 156 kB 00:00 2023-06-27T12:33:46.010 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:33:46.021 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:33:46.021 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:33:46.067 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:33:46.070 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:33:46.179 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:33:46.385 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4614.gba3fff72. 1/1 2023-06-27T12:33:46.811 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4614.gba3fff72. 1/1 2023-06-27T12:33:47.362 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4614.gba3fff72. 1/1 2023-06-27T12:33:47.362 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:47.362 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:33:47.362 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:33:47.362 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:47.362 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:33:47.575 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr 2023-06-27T12:33:48.413 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:04:47 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:33:49.587 INFO:teuthology.orchestra.run.smithi159.stdout:Package ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:33:49.739 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:33:49.740 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2023-06-27T12:33:49.741 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:33:49.835 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr-dashboard 2023-06-27T12:33:50.662 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:04:49 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:33:51.984 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:33:51.986 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:33:51.986 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:33:51.986 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:33:51.986 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:33:51.986 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 1.7 M 2023-06-27T12:33:51.986 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-06-27T12:33:51.986 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 23 k 2023-06-27T12:33:51.987 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 15 k 2023-06-27T12:33:51.987 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-06-27T12:33:51.987 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-27T12:33:51.987 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-27T12:33:51.987 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:51.987 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:33:51.987 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:33:51.988 INFO:teuthology.orchestra.run.smithi159.stdout:Install 6 Packages 2023-06-27T12:33:51.988 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:51.988 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 2.0 M 2023-06-27T12:33:51.988 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 8.2 M 2023-06-27T12:33:51.989 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:33:52.721 INFO:teuthology.orchestra.run.smithi159.stdout:(1/6): ceph-prometheus-alerts-18.0.0-4614.gba3f 94 kB/s | 15 kB 00:00 2023-06-27T12:33:52.780 INFO:teuthology.orchestra.run.smithi159.stdout:(2/6): ceph-grafana-dashboards-18.0.0-4614.gba3 108 kB/s | 23 kB 00:00 2023-06-27T12:33:52.846 INFO:teuthology.orchestra.run.smithi159.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 340 kB/s | 43 kB 00:00 2023-06-27T12:33:52.872 INFO:teuthology.orchestra.run.smithi159.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 367 kB/s | 34 kB 00:00 2023-06-27T12:33:52.905 INFO:teuthology.orchestra.run.smithi159.stdout:(5/6): ceph-mgr-dashboard-18.0.0-4614.gba3fff72 5.0 MB/s | 1.7 MB 00:00 2023-06-27T12:33:52.972 INFO:teuthology.orchestra.run.smithi159.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.5 MB/s | 196 kB 00:00 2023-06-27T12:33:52.974 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:33:52.974 INFO:teuthology.orchestra.run.smithi159.stdout:Total 2.0 MB/s | 2.0 MB 00:00 2023-06-27T12:33:52.985 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:33:52.995 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:33:52.995 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:33:53.078 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:33:53.081 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:33:53.338 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:33:53.524 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-06-27T12:33:53.751 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-06-27T12:33:53.928 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-06-27T12:33:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4614.gba3fff72.el8.n 4/6 2023-06-27T12:33:54.364 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4614.gba3fff72.el8. 5/6 2023-06-27T12:33:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4614.gba3fff72.el8.noarc 6/6 2023-06-27T12:33:54.819 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4614.gba3fff72.el8.noarc 6/6 2023-06-27T12:33:54.820 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4614.gba3fff72.el8. 1/6 2023-06-27T12:33:54.820 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4614.gba3fff72.el8.noarc 2/6 2023-06-27T12:33:54.820 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4614.gba3fff72.el8.n 3/6 2023-06-27T12:33:54.820 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-06-27T12:33:54.820 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-06-27T12:33:55.392 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-06-27T12:33:55.393 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:55.393 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:33:55.393 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:33:55.393 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:33:55.393 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:33:55.393 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-27T12:33:55.393 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-27T12:33:55.394 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-27T12:33:55.394 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:55.394 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:33:55.918 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr-diskprediction-local 2023-06-27T12:33:56.751 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:04:55 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:33:58.058 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:33:58.059 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================================ 2023-06-27T12:33:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:33:58.060 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================================ 2023-06-27T12:33:58.060 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:33:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 7.4 M 2023-06-27T12:33:58.060 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-06-27T12:33:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-06-27T12:33:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-06-27T12:33:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-06-27T12:33:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-06-27T12:33:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-06-27T12:33:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-06-27T12:33:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-06-27T12:33:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:58.062 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:33:58.062 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================================ 2023-06-27T12:33:58.062 INFO:teuthology.orchestra.run.smithi159.stdout:Install 8 Packages 2023-06-27T12:33:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:33:58.062 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 35 M 2023-06-27T12:33:58.063 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 210 M 2023-06-27T12:33:58.063 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:33:59.277 INFO:teuthology.orchestra.run.smithi159.stdout:(1/8): ceph-mgr-diskprediction-local-18.0.0-461 7.7 MB/s | 7.4 MB 00:00 2023-06-27T12:34:04.374 INFO:teuthology.orchestra.run.smithi138.stdout:(32/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 539 B/s | 103 kB 03:16 2023-06-27T12:34:04.378 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:34:04.378 INFO:teuthology.orchestra.run.smithi138.stdout:Total 873 kB/s | 175 MB 03:25 2023-06-27T12:34:04.454 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:34:04.492 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:34:04.493 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:34:05.357 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:34:05.360 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:34:06.963 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:34:07.210 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-06-27T12:34:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-06-27T12:34:07.645 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-06-27T12:34:07.860 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2023-06-27T12:34:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-06-27T12:34:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-06-27T12:34:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-06-27T12:34:08.542 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-06-27T12:34:08.717 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-06-27T12:34:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-06-27T12:34:09.191 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-06-27T12:34:09.415 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-06-27T12:34:09.778 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2023-06-27T12:34:10.015 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 14/32 2023-06-27T12:34:10.178 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-06-27T12:34:10.386 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-06-27T12:34:10.587 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-06-27T12:34:10.871 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-06-27T12:34:11.067 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-06-27T12:34:11.277 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-06-27T12:34:11.441 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-06-27T12:34:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-06-27T12:34:11.873 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-06-27T12:34:12.073 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-06-27T12:34:12.377 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-06-27T12:34:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-06-27T12:34:13.023 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4614.gba3fff72.el8. 27/32 2023-06-27T12:34:13.207 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 28/32 2023-06-27T12:34:30.483 INFO:teuthology.orchestra.run.smithi159.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 150 kB/s | 4.7 MB 00:32 2023-06-27T12:34:30.566 INFO:teuthology.orchestra.run.smithi159.stdout:(3/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 154 kB/s | 4.9 MB 00:32 2023-06-27T12:34:30.591 INFO:teuthology.orchestra.run.smithi159.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.0 MB/s | 225 kB 00:00 2023-06-27T12:34:30.825 INFO:teuthology.orchestra.run.smithi159.stdout:(5/8): libgfortran-8.5.0-20.el8.x86_64.rpm 2.7 MB/s | 645 kB 00:00 2023-06-27T12:34:30.900 INFO:teuthology.orchestra.run.smithi159.stdout:(6/8): libquadmath-8.5.0-20.el8.x86_64.rpm 2.3 MB/s | 172 kB 00:00 2023-06-27T12:34:31.367 INFO:teuthology.orchestra.run.smithi159.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 17 MB/s | 14 MB 00:00 2023-06-27T12:34:31.792 INFO:teuthology.orchestra.run.smithi159.stdout:(8/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 118 kB/s | 3.7 MB 00:32 2023-06-27T12:34:31.793 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:34:31.793 INFO:teuthology.orchestra.run.smithi159.stdout:Total 1.1 MB/s | 35 MB 00:33 2023-06-27T12:34:32.025 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:34:32.043 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:34:32.044 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:34:32.418 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:34:32.421 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:34:32.971 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:34:33.111 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2023-06-27T12:34:33.255 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2023-06-27T12:34:33.411 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2023-06-27T12:34:34.007 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2023-06-27T12:34:34.194 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-06-27T12:34:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-06-27T12:34:35.004 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-06-27T12:34:35.677 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-06-27T12:34:35.980 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-06-27T12:34:36.243 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-06-27T12:34:38.028 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-06-27T12:34:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-06-27T12:34:39.427 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4614.gba3fff7 8/8 2023-06-27T12:34:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4614.gba3fff7 8/8 2023-06-27T12:34:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4614.gba3fff7 1/8 2023-06-27T12:34:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-06-27T12:34:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-06-27T12:34:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-06-27T12:34:39.720 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-06-27T12:34:39.720 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-06-27T12:34:39.720 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2023-06-27T12:34:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2023-06-27T12:34:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:40.297 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:34:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:34:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-06-27T12:34:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-06-27T12:34:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.15-4.el8.x86_64 2023-06-27T12:34:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-06-27T12:34:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-27T12:34:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-27T12:34:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-06-27T12:34:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:40.298 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:34:40.420 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr-rook 2023-06-27T12:34:41.250 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:05:40 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:34:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 28/32 2023-06-27T12:34:41.502 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:18.0.0-4614.gba3fff72.el8.x86_64 29/32 2023-06-27T12:34:42.410 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:18.0.0-4614.gba3fff72.el8.x86_64 29/32 2023-06-27T12:34:42.580 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:34:42.582 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:34:42.582 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:34:42.582 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:34:42.583 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:34:42.583 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 47 k 2023-06-27T12:34:42.583 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-06-27T12:34:42.583 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-27T12:34:42.583 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-27T12:34:42.583 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-27T12:34:42.584 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-06-27T12:34:42.584 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-06-27T12:34:42.584 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-27T12:34:42.584 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-06-27T12:34:42.584 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-06-27T12:34:42.584 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-06-27T12:34:42.584 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-06-27T12:34:42.585 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-27T12:34:42.585 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-27T12:34:42.585 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:42.585 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:34:42.585 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:34:42.585 INFO:teuthology.orchestra.run.smithi159.stdout:Install 13 Packages 2023-06-27T12:34:42.585 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:42.586 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 1.9 M 2023-06-27T12:34:42.586 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 25 M 2023-06-27T12:34:42.586 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:34:42.598 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:18.0.0-4614.gba3fff72.el8.x86_64 30/32 2023-06-27T12:34:42.815 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:18.0.0-4614.gba3fff72.el8.x86_64 30/32 2023-06-27T12:34:42.956 INFO:teuthology.orchestra.run.smithi159.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 191 kB/s | 20 kB 00:00 2023-06-27T12:34:42.973 INFO:teuthology.orchestra.run.smithi159.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 219 kB/s | 27 kB 00:00 2023-06-27T12:34:42.990 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:18.0.0-4614.gba3fff72.el8.x86_64 31/32 2023-06-27T12:34:43.007 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:18.0.0-4614.gba3fff72.el8.x86_64 31/32 2023-06-27T12:34:43.065 INFO:teuthology.orchestra.run.smithi159.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-06-27T12:34:43.090 INFO:teuthology.orchestra.run.smithi159.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 941 kB/s | 110 kB 00:00 2023-06-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi159.stdout:(5/13): ceph-mgr-rook-18.0.0-4614.gba3fff72.el8 180 kB/s | 47 kB 00:00 2023-06-27T12:34:43.122 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:18.0.0-4614.gba3fff72.el8.x86_64 32/32 2023-06-27T12:34:43.208 INFO:teuthology.orchestra.run.smithi159.stdout:(6/13): python3-cachetools-3.1.1-4.el8.noarch.r 360 kB/s | 33 kB 00:00 2023-06-27T12:34:43.224 INFO:teuthology.orchestra.run.smithi159.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 318 kB/s | 43 kB 00:00 2023-06-27T12:34:43.258 INFO:teuthology.orchestra.run.smithi159.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 324 kB/s | 16 kB 00:00 2023-06-27T12:34:43.283 INFO:teuthology.orchestra.run.smithi159.stdout:(9/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 710 kB/s | 155 kB 00:00 2023-06-27T12:34:43.342 INFO:teuthology.orchestra.run.smithi159.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 694 kB/s | 81 kB 00:00 2023-06-27T12:34:43.392 INFO:teuthology.orchestra.run.smithi159.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 1.2 MB/s | 61 kB 00:00 2023-06-27T12:34:43.417 INFO:teuthology.orchestra.run.smithi159.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 460 kB/s | 61 kB 00:00 2023-06-27T12:34:43.484 INFO:teuthology.orchestra.run.smithi159.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 5.2 MB/s | 1.2 MB 00:00 2023-06-27T12:34:43.485 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:34:43.485 INFO:teuthology.orchestra.run.smithi159.stdout:Total 2.1 MB/s | 1.9 MB 00:00 2023-06-27T12:34:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-2:18.0.0-4614.gba3fff72.el8.x86_64 32/32 2023-06-27T12:34:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:18.0.0-4614.gba3fff72.el8.x86_64 1/32 2023-06-27T12:34:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:18.0.0-4614.gba3fff72.el8.x86_64 2/32 2023-06-27T12:34:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 3/32 2023-06-27T12:34:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:18.0.0-4614.gba3fff72.el8.x86_64 4/32 2023-06-27T12:34:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:18.0.0-4614.gba3fff72.el8.x86_64 5/32 2023-06-27T12:34:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4614.gba3fff72.el8. 6/32 2023-06-27T12:34:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-06-27T12:34:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-06-27T12:34:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-06-27T12:34:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-06-27T12:34:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-06-27T12:34:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2023-06-27T12:34:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 13/32 2023-06-27T12:34:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-06-27T12:34:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-06-27T12:34:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-06-27T12:34:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-06-27T12:34:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-06-27T12:34:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-06-27T12:34:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-06-27T12:34:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-06-27T12:34:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-06-27T12:34:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-06-27T12:34:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-06-27T12:34:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-06-27T12:34:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-06-27T12:34:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-06-27T12:34:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-06-27T12:34:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2023-06-27T12:34:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-06-27T12:34:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-06-27T12:34:43.537 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:34:43.552 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:34:43.553 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:34:43.637 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:34:43.640 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:34:44.117 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:34:44.346 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-06-27T12:34:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-06-27T12:34:44.616 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-06-27T12:34:44.617 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:34:44.617 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:34:44.617 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:34:44.617 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:34:44.617 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:34:44.617 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:34:44.617 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:34:44.618 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:34:44.618 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-27T12:34:44.618 INFO:teuthology.orchestra.run.smithi138.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-27T12:34:44.618 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-06-27T12:34:44.618 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-27T12:34:44.618 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-27T12:34:44.618 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-06-27T12:34:44.618 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-27T12:34:44.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-27T12:34:44.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-27T12:34:44.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.0.6-14.el8.noarch 2023-06-27T12:34:44.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-27T12:34:44.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-27T12:34:44.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-27T12:34:44.619 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-2.6-1.el8.noarch 2023-06-27T12:34:44.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-27T12:34:44.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-27T12:34:44.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-27T12:34:44.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-27T12:34:44.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-27T12:34:44.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-27T12:34:44.620 INFO:teuthology.orchestra.run.smithi138.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-27T12:34:44.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-27T12:34:44.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-27T12:34:44.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-27T12:34:44.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-27T12:34:44.621 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-27T12:34:44.621 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:34:44.621 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:34:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-06-27T12:34:44.771 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-base 2023-06-27T12:34:44.957 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-06-27T12:34:45.148 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-06-27T12:34:45.395 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-06-27T12:34:45.606 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:09:54 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:34:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-06-27T12:34:45.930 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-06-27T12:34:46.782 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-base-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:34:46.934 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:34:46.935 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-06-27T12:34:46.935 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:34:46.998 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-06-27T12:34:47.299 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-06-27T12:34:47.494 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-06-27T12:34:47.539 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install cephadm 2023-06-27T12:34:47.684 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-06-27T12:34:47.884 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:18.0.0-4614.gba3fff72.el8.noarch 13/13 2023-06-27T12:34:48.175 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4614.gba3fff72.el8.noarch 13/13 2023-06-27T12:34:48.175 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4614.gba3fff72.el8.noarch 1/13 2023-06-27T12:34:48.175 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-06-27T12:34:48.175 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-06-27T12:34:48.175 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-06-27T12:34:48.176 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-06-27T12:34:48.176 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-06-27T12:34:48.176 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-06-27T12:34:48.176 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-06-27T12:34:48.176 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-06-27T12:34:48.176 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-06-27T12:34:48.176 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-06-27T12:34:48.177 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-06-27T12:34:48.376 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:09:57 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:34:48.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-06-27T12:34:48.859 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:48.860 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:34:48.860 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:34:48.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-27T12:34:48.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-27T12:34:48.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-27T12:34:48.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-27T12:34:48.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-27T12:34:48.861 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-27T12:34:48.861 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-27T12:34:48.861 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-27T12:34:48.861 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-27T12:34:48.861 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-27T12:34:48.861 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-27T12:34:48.861 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-27T12:34:48.862 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:48.862 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:34:48.949 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr-cephadm 2023-06-27T12:34:49.716 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:34:49.717 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:34:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:34:49.717 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:34:49.717 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:34:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 166 k 2023-06-27T12:34:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:34:49.718 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:34:49.718 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:34:49.718 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2023-06-27T12:34:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:34:49.719 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 166 k 2023-06-27T12:34:49.719 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 654 k 2023-06-27T12:34:49.719 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:34:49.781 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:05:48 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:34:50.211 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm-18.0.0-4614.gba3fff72.el8.noarch.rpm 337 kB/s | 166 kB 00:00 2023-06-27T12:34:50.212 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:34:50.212 INFO:teuthology.orchestra.run.smithi138.stdout:Total 336 kB/s | 166 kB 00:00 2023-06-27T12:34:50.213 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:34:50.221 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:34:50.222 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:34:50.224 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:34:50.228 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:34:50.858 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:34:51.105 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:34:51.106 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:34:51.106 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:34:51.107 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:34:51.107 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:34:51.107 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 129 k 2023-06-27T12:34:51.108 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-06-27T12:34:51.108 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-27T12:34:51.108 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-06-27T12:34:51.108 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-06-27T12:34:51.109 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-27T12:34:51.109 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:51.109 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:34:51.110 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:34:51.110 INFO:teuthology.orchestra.run.smithi159.stdout:Install 5 Packages 2023-06-27T12:34:51.110 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:51.111 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 6.0 M 2023-06-27T12:34:51.112 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 26 M 2023-06-27T12:34:51.112 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:34:51.467 INFO:teuthology.orchestra.run.smithi159.stdout:(1/5): ceph-mgr-cephadm-18.0.0-4614.gba3fff72.e 737 kB/s | 129 kB 00:00 2023-06-27T12:34:51.525 INFO:teuthology.orchestra.run.smithi159.stdout:(2/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 2.2 MB/s | 538 kB 00:00 2023-06-27T12:34:51.776 INFO:teuthology.orchestra.run.smithi159.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 245 kB/s | 60 kB 00:00 2023-06-27T12:34:51.826 INFO:teuthology.orchestra.run.smithi159.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.3 MB/s | 476 kB 00:00 2023-06-27T12:34:51.959 INFO:teuthology.orchestra.run.smithi159.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 7.2 MB/s | 4.8 MB 00:00 2023-06-27T12:34:51.962 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:34:51.962 INFO:teuthology.orchestra.run.smithi159.stdout:Total 7.0 MB/s | 6.0 MB 00:00 2023-06-27T12:34:52.014 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:34:52.025 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:34:52.025 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:34:52.127 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:34:52.130 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:34:52.410 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:34:52.733 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-06-27T12:34:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-06-27T12:34:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-06-27T12:34:54.314 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-06-27T12:34:54.524 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 5/5 2023-06-27T12:34:54.814 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 5/5 2023-06-27T12:34:54.815 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/5 2023-06-27T12:34:54.815 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-06-27T12:34:54.815 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-06-27T12:34:54.815 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-06-27T12:34:55.367 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-06-27T12:34:55.368 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:55.368 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:34:55.368 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:34:55.368 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-27T12:34:55.368 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-27T12:34:55.368 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-27T12:34:55.369 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-27T12:34:55.369 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:55.369 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:34:55.470 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-fuse 2023-06-27T12:34:56.300 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:05:55 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:34:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:34:57.625 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:34:57.626 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:34:57.626 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:34:57.626 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:34:57.626 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:34:57.626 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 819 k 2023-06-27T12:34:57.626 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-06-27T12:34:57.627 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-06-27T12:34:57.627 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:57.627 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:34:57.627 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:34:57.627 INFO:teuthology.orchestra.run.smithi159.stdout:Install 2 Packages 2023-06-27T12:34:57.627 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:57.628 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 902 k 2023-06-27T12:34:57.628 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 3.3 M 2023-06-27T12:34:57.628 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:34:57.959 INFO:teuthology.orchestra.run.smithi159.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 556 kB/s | 83 kB 00:00 2023-06-27T12:34:58.059 INFO:teuthology.orchestra.run.smithi159.stdout:(2/2): ceph-fuse-18.0.0-4614.gba3fff72.el8.x86_ 3.2 MB/s | 819 kB 00:00 2023-06-27T12:34:58.060 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:34:58.060 INFO:teuthology.orchestra.run.smithi159.stdout:Total 2.0 MB/s | 902 kB 00:00 2023-06-27T12:34:58.069 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:34:58.091 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:34:58.091 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:34:58.177 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:34:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:34:58.346 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:34:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2023-06-27T12:34:58.790 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 2/2 2023-06-27T12:34:59.172 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 2/2 2023-06-27T12:34:59.172 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 1/2 2023-06-27T12:34:59.479 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:34:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:34:59.733 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2023-06-27T12:34:59.733 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:59.733 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:34:59.734 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2023-06-27T12:34:59.734 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:34:59.734 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:34:59.927 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-volume 2023-06-27T12:35:00.762 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:05:59 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:01.055 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:35:01.055 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:01.055 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:35:01.055 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:35:01.055 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:01.056 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:35:01.350 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-immutable-object-cache 2023-06-27T12:35:02.060 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:02.061 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:02.061 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-06-27T12:35:02.061 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:02.061 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:35:02.061 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 258 k 2023-06-27T12:35:02.062 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:02.062 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:35:02.062 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:02.062 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2023-06-27T12:35:02.062 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:02.062 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 258 k 2023-06-27T12:35:02.062 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 1.2 M 2023-06-27T12:35:02.063 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:35:02.196 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:10:11 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:35:02.413 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-volume-18.0.0-4614.gba3fff72.el8.noarch.rp 736 kB/s | 258 kB 00:00 2023-06-27T12:35:02.413 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:35:02.414 INFO:teuthology.orchestra.run.smithi159.stdout:Total 733 kB/s | 258 kB 00:00 2023-06-27T12:35:02.414 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:35:02.424 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:35:02.424 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:35:02.454 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:35:02.457 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:35:02.634 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:35:02.932 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:35:03.336 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:35:03.524 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:35:03.525 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:35:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2023-06-27T12:35:03.526 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:35:03.526 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:35:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 156 k 2023-06-27T12:35:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:03.526 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:35:03.526 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:35:03.527 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2023-06-27T12:35:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:03.527 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 156 k 2023-06-27T12:35:03.527 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 540 k 2023-06-27T12:35:03.527 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:35:03.836 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-immutable-object-cache-18.0.0-4614.gba3fff 505 kB/s | 156 kB 00:00 2023-06-27T12:35:03.837 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:35:03.837 INFO:teuthology.orchestra.run.smithi138.stdout:Total 503 kB/s | 156 kB 00:00 2023-06-27T12:35:03.837 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:35:03.847 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:35:03.848 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:35:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:35:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:03.884 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:35:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:35:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:03.884 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:03.894 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:35:03.897 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:35:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:35:04.091 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librados-devel 2023-06-27T12:35:04.178 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4614.gba3fff72. 1/1 2023-06-27T12:35:04.593 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4614.gba3fff72. 1/1 2023-06-27T12:35:04.929 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:03 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:05.125 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4614.gba3fff72. 1/1 2023-06-27T12:35:05.125 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:05.125 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:35:05.125 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:35:05.126 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:05.126 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:35:05.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr 2023-06-27T12:35:06.142 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:10:15 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:35:06.243 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:06.244 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:06.245 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repo Size 2023-06-27T12:35:06.245 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:06.245 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:35:06.245 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 139 k 2023-06-27T12:35:06.245 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:06.246 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:35:06.246 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:06.246 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2023-06-27T12:35:06.246 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:06.246 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 139 k 2023-06-27T12:35:06.246 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 512 k 2023-06-27T12:35:06.246 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:35:06.434 INFO:teuthology.orchestra.run.smithi159.stdout:librados-devel-18.0.0-4614.gba3fff72.el8.x86_64 736 kB/s | 139 kB 00:00 2023-06-27T12:35:06.435 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:35:06.435 INFO:teuthology.orchestra.run.smithi159.stdout:Total 731 kB/s | 139 kB 00:00 2023-06-27T12:35:06.435 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:35:06.440 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:35:06.440 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:35:06.490 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:35:06.493 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:35:06.611 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:35:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:07.104 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:07.327 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-mgr-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:35:07.480 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:35:07.481 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-06-27T12:35:07.481 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:35:07.576 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-dashboard 2023-06-27T12:35:07.666 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:07.667 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:35:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:35:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:07.667 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:07.898 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install libcephfs2 2023-06-27T12:35:08.397 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:10:17 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:35:08.750 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:07 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:09.719 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:35:09.720 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:35:09.720 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:35:09.720 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:35:09.721 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:35:09.721 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 1.7 M 2023-06-27T12:35:09.721 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-06-27T12:35:09.721 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 23 k 2023-06-27T12:35:09.721 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 15 k 2023-06-27T12:35:09.721 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-06-27T12:35:09.721 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-27T12:35:09.722 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-27T12:35:09.722 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:09.722 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:35:09.722 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:35:09.722 INFO:teuthology.orchestra.run.smithi138.stdout:Install 6 Packages 2023-06-27T12:35:09.722 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:09.723 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.0 M 2023-06-27T12:35:09.723 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 8.2 M 2023-06-27T12:35:09.723 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:35:09.918 INFO:teuthology.orchestra.run.smithi159.stdout:Package libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:35:10.070 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:10.071 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2023-06-27T12:35:10.071 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:10.182 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install libcephfs-devel 2023-06-27T12:35:10.266 INFO:teuthology.orchestra.run.smithi138.stdout:(1/6): ceph-grafana-dashboards-18.0.0-4614.gba3 122 kB/s | 23 kB 00:00 2023-06-27T12:35:10.283 INFO:teuthology.orchestra.run.smithi138.stdout:(2/6): ceph-prometheus-alerts-18.0.0-4614.gba3f 71 kB/s | 15 kB 00:00 2023-06-27T12:35:10.400 INFO:teuthology.orchestra.run.smithi138.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 318 kB/s | 43 kB 00:00 2023-06-27T12:35:10.425 INFO:teuthology.orchestra.run.smithi138.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 237 kB/s | 34 kB 00:00 2023-06-27T12:35:10.776 INFO:teuthology.orchestra.run.smithi138.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 523 kB/s | 196 kB 00:00 2023-06-27T12:35:11.018 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:10 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:11.684 INFO:teuthology.orchestra.run.smithi138.stdout:(6/6): ceph-mgr-dashboard-18.0.0-4614.gba3fff72 1.1 MB/s | 1.7 MB 00:01 2023-06-27T12:35:11.686 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:35:11.686 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.0 MB/s | 2.0 MB 00:01 2023-06-27T12:35:11.698 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:35:11.708 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:35:11.709 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:35:11.794 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:35:11.797 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:35:12.054 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:35:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-06-27T12:35:12.331 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:12.332 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:12.332 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repo Size 2023-06-27T12:35:12.332 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:12.332 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:35:12.332 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 31 k 2023-06-27T12:35:12.333 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:12.333 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:35:12.333 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:12.333 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2023-06-27T12:35:12.333 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:12.333 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 31 k 2023-06-27T12:35:12.333 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 139 k 2023-06-27T12:35:12.334 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:35:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-06-27T12:35:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-06-27T12:35:12.708 INFO:teuthology.orchestra.run.smithi159.stdout:libcephfs-devel-18.0.0-4614.gba3fff72.el8.x86_6 83 kB/s | 31 kB 00:00 2023-06-27T12:35:12.709 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:35:12.709 INFO:teuthology.orchestra.run.smithi159.stdout:Total 82 kB/s | 31 kB 00:00 2023-06-27T12:35:12.709 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:35:12.712 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:35:12.713 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:35:12.733 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:35:12.736 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:35:12.774 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4614.gba3fff72.el8.n 4/6 2023-06-27T12:35:12.798 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:35:12.982 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4614.gba3fff72.el8. 5/6 2023-06-27T12:35:13.229 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:13.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4614.gba3fff72.el8.noarc 6/6 2023-06-27T12:35:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4614.gba3fff72.el8.noarc 6/6 2023-06-27T12:35:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4614.gba3fff72.el8. 1/6 2023-06-27T12:35:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4614.gba3fff72.el8.noarc 2/6 2023-06-27T12:35:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4614.gba3fff72.el8.n 3/6 2023-06-27T12:35:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-06-27T12:35:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-06-27T12:35:13.735 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:13.735 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:13.735 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:35:13.736 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:35:13.736 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:13.736 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:13.841 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librados2 2023-06-27T12:35:14.174 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-06-27T12:35:14.175 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:14.175 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:35:14.175 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:35:14.176 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:35:14.176 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:35:14.176 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-27T12:35:14.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-27T12:35:14.177 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-27T12:35:14.177 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:14.177 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:35:14.288 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-diskprediction-local 2023-06-27T12:35:14.681 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:13 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:10:24 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:35:15.861 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:35:16.014 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:16.015 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2023-06-27T12:35:16.015 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:16.106 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librbd1 2023-06-27T12:35:16.438 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:35:16.439 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================================ 2023-06-27T12:35:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:35:16.439 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================================ 2023-06-27T12:35:16.439 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:35:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 7.4 M 2023-06-27T12:35:16.440 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-06-27T12:35:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-06-27T12:35:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-06-27T12:35:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-06-27T12:35:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-06-27T12:35:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-06-27T12:35:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-06-27T12:35:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-06-27T12:35:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:16.441 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:35:16.441 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================================ 2023-06-27T12:35:16.441 INFO:teuthology.orchestra.run.smithi138.stdout:Install 8 Packages 2023-06-27T12:35:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:16.442 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 35 M 2023-06-27T12:35:16.442 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 210 M 2023-06-27T12:35:16.442 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:35:16.938 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:15 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:17.399 INFO:teuthology.orchestra.run.smithi138.stdout:(1/8): ceph-mgr-diskprediction-local-18.0.0-461 10 MB/s | 7.4 MB 00:00 2023-06-27T12:35:18.099 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:35:18.253 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:18.254 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2023-06-27T12:35:18.254 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:18.346 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-rados 2023-06-27T12:35:19.176 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:18 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:20.338 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-rados-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:35:20.491 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:20.492 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2023-06-27T12:35:20.493 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:20.579 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-rgw 2023-06-27T12:35:21.412 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:20 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:22.579 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-rgw-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:35:22.732 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:22.734 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2023-06-27T12:35:22.734 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:22.819 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-cephfs 2023-06-27T12:35:23.645 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:22 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:24.806 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-cephfs-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:35:24.958 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:24.959 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2023-06-27T12:35:24.960 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:25.044 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-rbd 2023-06-27T12:35:25.875 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:24 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:27.051 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-rbd-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:35:27.205 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:27.206 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2023-06-27T12:35:27.206 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:27.293 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install rbd-fuse 2023-06-27T12:35:28.119 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:27 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:29.444 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:29.445 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:29.445 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2023-06-27T12:35:29.445 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:35:29.446 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 91 k 2023-06-27T12:35:29.446 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:35:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2023-06-27T12:35:29.446 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 91 k 2023-06-27T12:35:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 303 k 2023-06-27T12:35:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:35:29.625 INFO:teuthology.orchestra.run.smithi159.stdout:rbd-fuse-18.0.0-4614.gba3fff72.el8.x86_64.rpm 512 kB/s | 91 kB 00:00 2023-06-27T12:35:29.626 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:35:29.626 INFO:teuthology.orchestra.run.smithi159.stdout:Total 508 kB/s | 91 kB 00:00 2023-06-27T12:35:29.626 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:35:29.635 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:35:29.636 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:35:29.684 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:35:29.687 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:35:29.788 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:35:29.963 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:30.238 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:30.789 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:30.790 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:30.790 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:35:30.790 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:35:30.790 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:30.790 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:30.991 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install rbd-mirror 2023-06-27T12:35:31.829 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:30 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:33.106 INFO:teuthology.orchestra.run.smithi138.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 294 kB/s | 4.7 MB 00:16 2023-06-27T12:35:33.148 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:33.149 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:33.149 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repo Size 2023-06-27T12:35:33.149 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:33.150 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:35:33.150 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 3.3 M 2023-06-27T12:35:33.150 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:33.150 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:35:33.151 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:33.151 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2023-06-27T12:35:33.151 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:33.151 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 3.3 M 2023-06-27T12:35:33.151 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 18 M 2023-06-27T12:35:33.151 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:35:33.181 INFO:teuthology.orchestra.run.smithi138.stdout:(3/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 301 kB/s | 4.9 MB 00:16 2023-06-27T12:35:33.240 INFO:teuthology.orchestra.run.smithi138.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 241 kB/s | 3.7 MB 00:15 2023-06-27T12:35:33.274 INFO:teuthology.orchestra.run.smithi138.stdout:(5/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.3 MB/s | 225 kB 00:00 2023-06-27T12:35:33.458 INFO:teuthology.orchestra.run.smithi138.stdout:(6/8): libquadmath-8.5.0-20.el8.x86_64.rpm 939 kB/s | 172 kB 00:00 2023-06-27T12:35:33.519 INFO:teuthology.orchestra.run.smithi159.stdout:rbd-mirror-18.0.0-4614.gba3fff72.el8.x86_64.rpm 9.0 MB/s | 3.3 MB 00:00 2023-06-27T12:35:33.519 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:35:33.520 INFO:teuthology.orchestra.run.smithi159.stdout:Total 9.0 MB/s | 3.3 MB 00:00 2023-06-27T12:35:33.520 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:35:33.531 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:35:33.531 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:35:33.566 INFO:teuthology.orchestra.run.smithi138.stdout:(7/8): libgfortran-8.5.0-20.el8.x86_64.rpm 1.9 MB/s | 645 kB 00:00 2023-06-27T12:35:33.592 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:35:33.595 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:35:34.040 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:35:34.167 INFO:teuthology.orchestra.run.smithi138.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 14 MB/s | 14 MB 00:00 2023-06-27T12:35:34.167 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:35:34.167 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.0 MB/s | 35 MB 00:17 2023-06-27T12:35:34.219 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:34.401 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:35:34.419 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:35:34.419 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:35:34.642 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:34.808 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:35:34.811 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:35:35.203 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:35.203 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:35.203 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:35:35.203 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:35:35.203 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:35.204 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:35:35.417 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install rbd-nbd 2023-06-27T12:35:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2023-06-27T12:35:35.636 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2023-06-27T12:35:35.806 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2023-06-27T12:35:36.254 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:06:35 ago on Tue 27 Jun 2023 12:29:01 PM UTC. 2023-06-27T12:35:36.415 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2023-06-27T12:35:36.631 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-06-27T12:35:37.243 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-06-27T12:35:37.404 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-06-27T12:35:37.577 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-06-27T12:35:37.578 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:37.578 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2023-06-27T12:35:37.579 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:37.579 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-06-27T12:35:37.579 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 177 k 2023-06-27T12:35:37.579 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:37.579 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-06-27T12:35:37.579 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-06-27T12:35:37.579 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2023-06-27T12:35:37.580 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:37.580 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 177 k 2023-06-27T12:35:37.580 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 626 k 2023-06-27T12:35:37.581 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-06-27T12:35:37.764 INFO:teuthology.orchestra.run.smithi159.stdout:rbd-nbd-18.0.0-4614.gba3fff72.el8.x86_64.rpm 957 kB/s | 177 kB 00:00 2023-06-27T12:35:37.765 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:35:37.765 INFO:teuthology.orchestra.run.smithi159.stdout:Total 950 kB/s | 177 kB 00:00 2023-06-27T12:35:37.765 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-06-27T12:35:37.775 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-06-27T12:35:37.776 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-06-27T12:35:37.825 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-06-27T12:35:37.828 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-06-27T12:35:37.938 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-06-27T12:35:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-06-27T12:35:38.141 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:38.393 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-06-27T12:35:38.438 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-nbd-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-06-27T12:35:38.998 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:35:38.998 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:38.998 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-06-27T12:35:38.999 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:35:38.999 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:35:38.999 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-06-27T12:35:39.191 DEBUG:teuthology.parallel:result is None 2023-06-27T12:35:40.432 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-06-27T12:35:41.686 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-06-27T12:35:41.897 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4614.gba3fff7 8/8 2023-06-27T12:35:42.211 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4614.gba3fff7 8/8 2023-06-27T12:35:42.211 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4614.gba3fff7 1/8 2023-06-27T12:35:42.211 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-06-27T12:35:42.212 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-06-27T12:35:42.212 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-06-27T12:35:42.212 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-06-27T12:35:42.212 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-06-27T12:35:42.212 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2023-06-27T12:35:42.802 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2023-06-27T12:35:42.804 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:42.804 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:35:42.804 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:35:42.804 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-06-27T12:35:42.805 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-06-27T12:35:42.805 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.15-4.el8.x86_64 2023-06-27T12:35:42.805 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-06-27T12:35:42.805 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-27T12:35:42.805 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-27T12:35:42.805 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-06-27T12:35:42.805 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:42.805 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:35:42.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-rook 2023-06-27T12:35:43.757 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:10:52 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:35:45.080 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:35:45.082 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:35:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:35:45.082 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:35:45.082 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:35:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 47 k 2023-06-27T12:35:45.083 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-06-27T12:35:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-27T12:35:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-27T12:35:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-27T12:35:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-06-27T12:35:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-06-27T12:35:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-27T12:35:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-06-27T12:35:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-06-27T12:35:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-06-27T12:35:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-06-27T12:35:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-27T12:35:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-27T12:35:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:45.085 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:35:45.085 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:35:45.085 INFO:teuthology.orchestra.run.smithi138.stdout:Install 13 Packages 2023-06-27T12:35:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:35:45.087 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 1.9 M 2023-06-27T12:35:45.087 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 25 M 2023-06-27T12:35:45.087 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:35:45.561 INFO:teuthology.orchestra.run.smithi138.stdout:(1/13): ceph-mgr-rook-18.0.0-4614.gba3fff72.el8 272 kB/s | 47 kB 00:00 2023-06-27T12:36:17.404 INFO:teuthology.orchestra.run.smithi138.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 637 B/s | 20 kB 00:32 2023-06-27T12:36:17.421 INFO:teuthology.orchestra.run.smithi138.stdout:(3/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 848 B/s | 27 kB 00:32 2023-06-27T12:36:17.462 INFO:teuthology.orchestra.run.smithi138.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 4.0 kB/s | 126 kB 00:31 2023-06-27T12:36:17.513 INFO:teuthology.orchestra.run.smithi138.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.0 MB/s | 110 kB 00:00 2023-06-27T12:36:17.622 INFO:teuthology.orchestra.run.smithi138.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 771 kB/s | 155 kB 00:00 2023-06-27T12:36:17.655 INFO:teuthology.orchestra.run.smithi138.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 221 kB/s | 43 kB 00:00 2023-06-27T12:36:17.681 INFO:teuthology.orchestra.run.smithi138.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 197 kB/s | 33 kB 00:00 2023-06-27T12:36:17.781 INFO:teuthology.orchestra.run.smithi138.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 103 kB/s | 16 kB 00:00 2023-06-27T12:36:17.840 INFO:teuthology.orchestra.run.smithi138.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 442 kB/s | 81 kB 00:00 2023-06-27T12:36:17.990 INFO:teuthology.orchestra.run.smithi138.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 408 kB/s | 61 kB 00:00 2023-06-27T12:36:18.040 INFO:teuthology.orchestra.run.smithi138.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 238 kB/s | 61 kB 00:00 2023-06-27T12:36:18.090 INFO:teuthology.orchestra.run.smithi138.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.8 MB/s | 1.2 MB 00:00 2023-06-27T12:36:18.091 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:36:18.091 INFO:teuthology.orchestra.run.smithi138.stdout:Total 60 kB/s | 1.9 MB 00:33 2023-06-27T12:36:18.146 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:36:18.162 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:36:18.162 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:36:18.249 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:36:18.252 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:36:18.744 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:36:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-06-27T12:36:19.174 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-06-27T12:36:19.418 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-06-27T12:36:19.639 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-06-27T12:36:19.846 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-06-27T12:36:20.078 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-06-27T12:36:20.344 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-06-27T12:36:20.595 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-06-27T12:36:21.603 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-06-27T12:36:21.881 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-06-27T12:36:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-06-27T12:36:22.274 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-06-27T12:36:22.449 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:18.0.0-4614.gba3fff72.el8.noarch 13/13 2023-06-27T12:36:22.742 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4614.gba3fff72.el8.noarch 13/13 2023-06-27T12:36:22.742 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4614.gba3fff72.el8.noarch 1/13 2023-06-27T12:36:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-06-27T12:36:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-06-27T12:36:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-06-27T12:36:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-06-27T12:36:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-06-27T12:36:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-06-27T12:36:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-06-27T12:36:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-06-27T12:36:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-06-27T12:36:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-06-27T12:36:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-06-27T12:36:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-06-27T12:36:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:23.435 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:36:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:36:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-27T12:36:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-27T12:36:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-27T12:36:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-27T12:36:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-27T12:36:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-27T12:36:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-27T12:36:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-27T12:36:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-27T12:36:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-27T12:36:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-27T12:36:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-27T12:36:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:23.438 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:23.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-cephadm 2023-06-27T12:36:24.354 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:11:33 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:25.669 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:25.670 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:25.670 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:36:25.671 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:25.671 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:36:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 129 k 2023-06-27T12:36:25.671 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-06-27T12:36:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-27T12:36:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-06-27T12:36:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-06-27T12:36:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-27T12:36:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:25.672 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:36:25.672 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:25.672 INFO:teuthology.orchestra.run.smithi138.stdout:Install 5 Packages 2023-06-27T12:36:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:25.673 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 6.0 M 2023-06-27T12:36:25.673 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 26 M 2023-06-27T12:36:25.673 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:36:26.142 INFO:teuthology.orchestra.run.smithi138.stdout:(1/5): ceph-mgr-cephadm-18.0.0-4614.gba3fff72.e 705 kB/s | 129 kB 00:00 2023-06-27T12:36:26.168 INFO:teuthology.orchestra.run.smithi138.stdout:(2/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 2.5 MB/s | 538 kB 00:00 2023-06-27T12:36:26.327 INFO:teuthology.orchestra.run.smithi138.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 381 kB/s | 60 kB 00:00 2023-06-27T12:36:26.418 INFO:teuthology.orchestra.run.smithi138.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.7 MB/s | 476 kB 00:00 2023-06-27T12:36:26.802 INFO:teuthology.orchestra.run.smithi138.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 5.7 MB/s | 4.8 MB 00:00 2023-06-27T12:36:26.803 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:36:26.803 INFO:teuthology.orchestra.run.smithi138.stdout:Total 5.3 MB/s | 6.0 MB 00:01 2023-06-27T12:36:26.855 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:36:26.867 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:36:26.867 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:36:26.973 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:36:26.976 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:36:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:36:27.561 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-06-27T12:36:28.491 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-06-27T12:36:28.818 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-06-27T12:36:29.097 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-06-27T12:36:29.323 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 5/5 2023-06-27T12:36:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 5/5 2023-06-27T12:36:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 1/5 2023-06-27T12:36:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-06-27T12:36:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-06-27T12:36:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-06-27T12:36:30.187 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-06-27T12:36:30.187 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:30.187 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:36:30.187 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:36:30.187 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-27T12:36:30.188 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-27T12:36:30.188 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-27T12:36:30.188 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-27T12:36:30.188 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:30.188 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:30.294 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-fuse 2023-06-27T12:36:31.125 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:11:40 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:32.440 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:32.441 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:32.441 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:36:32.441 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:32.441 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:36:32.442 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 819 k 2023-06-27T12:36:32.442 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2023-06-27T12:36:32.442 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-06-27T12:36:32.442 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:32.442 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:36:32.442 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:32.442 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2023-06-27T12:36:32.442 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:32.443 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 902 k 2023-06-27T12:36:32.443 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 3.3 M 2023-06-27T12:36:32.443 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:36:32.786 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 512 kB/s | 83 kB 00:00 2023-06-27T12:36:32.903 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): ceph-fuse-18.0.0-4614.gba3fff72.el8.x86_ 2.9 MB/s | 819 kB 00:00 2023-06-27T12:36:32.904 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:36:32.904 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.9 MB/s | 902 kB 00:00 2023-06-27T12:36:32.914 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:36:32.935 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:36:32.936 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:36:33.023 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:36:33.026 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:36:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:36:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2023-06-27T12:36:33.630 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 2/2 2023-06-27T12:36:34.022 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 2/2 2023-06-27T12:36:34.022 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 1/2 2023-06-27T12:36:34.628 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2023-06-27T12:36:34.629 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:34.629 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:36:34.629 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2023-06-27T12:36:34.629 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:34.629 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:34.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-volume 2023-06-27T12:36:35.668 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:11:44 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:36.995 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:36.995 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:36.995 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2023-06-27T12:36:36.996 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:36.996 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:36:36.996 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:18.0.0-4614.gba3fff72.el8 ceph-noarch 258 k 2023-06-27T12:36:36.996 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:36.996 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:36:36.996 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:36.996 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2023-06-27T12:36:36.997 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:36.997 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 258 k 2023-06-27T12:36:36.997 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 1.2 M 2023-06-27T12:36:36.998 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:36:37.202 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-volume-18.0.0-4614.gba3fff72.el8.noarch.rp 1.2 MB/s | 258 kB 00:00 2023-06-27T12:36:37.202 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:36:37.203 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.2 MB/s | 258 kB 00:00 2023-06-27T12:36:37.203 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:36:37.213 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:36:37.213 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:36:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:36:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:36:37.433 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:36:37.663 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:36:38.079 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:36:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:18.0.0-4614.gba3fff72.el8.noarch 1/1 2023-06-27T12:36:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:38.634 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:36:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:18.0.0-4614.gba3fff72.el8.noarch 2023-06-27T12:36:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:38.634 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:38.819 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados-devel 2023-06-27T12:36:39.652 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:11:48 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:40.963 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:40.964 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:40.964 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2023-06-27T12:36:40.964 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:40.964 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:36:40.964 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 139 k 2023-06-27T12:36:40.964 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:40.965 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:36:40.965 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:40.965 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2023-06-27T12:36:40.965 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:40.965 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 139 k 2023-06-27T12:36:40.965 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 512 k 2023-06-27T12:36:40.966 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:36:41.142 INFO:teuthology.orchestra.run.smithi138.stdout:librados-devel-18.0.0-4614.gba3fff72.el8.x86_64 784 kB/s | 139 kB 00:00 2023-06-27T12:36:41.142 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:36:41.143 INFO:teuthology.orchestra.run.smithi138.stdout:Total 779 kB/s | 139 kB 00:00 2023-06-27T12:36:41.143 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:36:41.148 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:36:41.148 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:36:41.197 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:36:41.200 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:36:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:36:41.536 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:36:41.812 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:36:42.377 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:36:42.377 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:42.377 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:36:42.378 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:36:42.378 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:42.378 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:42.586 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs2 2023-06-27T12:36:43.425 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:11:52 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:44.594 INFO:teuthology.orchestra.run.smithi138.stdout:Package libcephfs2-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:36:44.745 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:44.746 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-06-27T12:36:44.746 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:44.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs-devel 2023-06-27T12:36:45.660 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:11:54 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:46.979 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:46.979 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:46.980 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2023-06-27T12:36:46.980 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:46.980 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:36:46.980 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 31 k 2023-06-27T12:36:46.980 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:46.980 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:36:46.981 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:36:46.981 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2023-06-27T12:36:46.981 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:46.981 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 31 k 2023-06-27T12:36:46.982 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 139 k 2023-06-27T12:36:46.982 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:36:47.197 INFO:teuthology.orchestra.run.smithi138.stdout:libcephfs-devel-18.0.0-4614.gba3fff72.el8.x86_6 143 kB/s | 31 kB 00:00 2023-06-27T12:36:47.198 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:36:47.198 INFO:teuthology.orchestra.run.smithi138.stdout:Total 142 kB/s | 31 kB 00:00 2023-06-27T12:36:47.199 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:36:47.202 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:36:47.202 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:36:47.228 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:36:47.231 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:36:47.288 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:36:47.487 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:36:47.754 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:36:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:36:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:48.292 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:36:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:36:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:36:48.293 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:48.407 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados2 2023-06-27T12:36:49.233 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:11:58 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:50.396 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:36:50.548 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:50.550 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-06-27T12:36:50.550 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:50.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librbd1 2023-06-27T12:36:51.468 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:12:00 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:52.627 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:36:52.779 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:52.780 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-06-27T12:36:52.780 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:52.874 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rados 2023-06-27T12:36:53.705 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:12:02 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:54.874 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rados-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:36:55.027 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:55.028 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-06-27T12:36:55.028 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:55.117 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rgw 2023-06-27T12:36:55.957 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:12:04 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:57.127 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rgw-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:36:57.282 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:57.283 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-06-27T12:36:57.284 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:57.377 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-cephfs 2023-06-27T12:36:58.212 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:12:07 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:36:59.395 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-cephfs-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:36:59.549 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:36:59.551 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-06-27T12:36:59.551 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:36:59.643 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rbd 2023-06-27T12:37:00.472 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:12:09 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:37:01.647 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rbd-2:18.0.0-4614.gba3fff72.el8.x86_64 is already installed. 2023-06-27T12:37:01.802 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:37:01.803 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2023-06-27T12:37:01.803 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:37:01.886 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-fuse 2023-06-27T12:37:02.711 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:12:11 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:37:04.032 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:37:04.033 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:37:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2023-06-27T12:37:04.033 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:37:04.033 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:37:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 91 k 2023-06-27T12:37:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:04.034 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:37:04.034 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:37:04.034 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2023-06-27T12:37:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:04.034 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 91 k 2023-06-27T12:37:04.034 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 303 k 2023-06-27T12:37:04.035 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:37:04.341 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-fuse-18.0.0-4614.gba3fff72.el8.x86_64.rpm 297 kB/s | 91 kB 00:00 2023-06-27T12:37:04.341 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:37:04.341 INFO:teuthology.orchestra.run.smithi138.stdout:Total 295 kB/s | 91 kB 00:00 2023-06-27T12:37:04.342 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:37:04.351 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:37:04.352 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:37:04.400 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:37:04.403 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:37:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:37:04.667 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:37:04.964 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:37:05.522 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:37:05.522 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:05.523 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:37:05.523 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:37:05.523 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:05.523 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:37:05.699 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-mirror 2023-06-27T12:37:06.534 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:12:15 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:37:07.879 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:37:07.880 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:37:07.880 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2023-06-27T12:37:07.880 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:37:07.880 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:37:07.881 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 3.3 M 2023-06-27T12:37:07.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:07.881 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:37:07.881 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:37:07.881 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2023-06-27T12:37:07.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:07.882 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.3 M 2023-06-27T12:37:07.882 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 18 M 2023-06-27T12:37:07.882 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:37:08.652 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-mirror-18.0.0-4614.gba3fff72.el8.x86_64.rpm 4.3 MB/s | 3.3 MB 00:00 2023-06-27T12:37:08.653 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:37:08.653 INFO:teuthology.orchestra.run.smithi138.stdout:Total 4.3 MB/s | 3.3 MB 00:00 2023-06-27T12:37:08.654 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:37:08.665 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:37:08.665 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:37:08.727 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:37:08.730 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:37:09.173 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:37:09.392 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:37:09.797 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:37:10.371 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:37:10.371 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:10.371 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:37:10.372 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:37:10.372 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:10.372 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:37:10.557 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-nbd 2023-06-27T12:37:11.393 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:12:20 ago on Tue 27 Jun 2023 12:24:51 PM UTC. 2023-06-27T12:37:12.715 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2023-06-27T12:37:12.715 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:37:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2023-06-27T12:37:12.716 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:37:12.716 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2023-06-27T12:37:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:18.0.0-4614.gba3fff72.el8 ceph 177 k 2023-06-27T12:37:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:12.716 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2023-06-27T12:37:12.717 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2023-06-27T12:37:12.717 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2023-06-27T12:37:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:12.717 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 177 k 2023-06-27T12:37:12.718 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 626 k 2023-06-27T12:37:12.718 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2023-06-27T12:37:13.013 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-nbd-18.0.0-4614.gba3fff72.el8.x86_64.rpm 596 kB/s | 177 kB 00:00 2023-06-27T12:37:13.014 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2023-06-27T12:37:13.014 INFO:teuthology.orchestra.run.smithi138.stdout:Total 593 kB/s | 177 kB 00:00 2023-06-27T12:37:13.015 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2023-06-27T12:37:13.025 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2023-06-27T12:37:13.025 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2023-06-27T12:37:13.075 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2023-06-27T12:37:13.078 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2023-06-27T12:37:13.204 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2023-06-27T12:37:13.373 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:37:13.658 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-nbd-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:37:14.254 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:18.0.0-4614.gba3fff72.el8.x86_64 1/1 2023-06-27T12:37:14.254 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:14.255 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2023-06-27T12:37:14.255 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:18.0.0-4614.gba3fff72.el8.x86_64 2023-06-27T12:37:14.255 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:14.255 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2023-06-27T12:37:14.441 DEBUG:teuthology.parallel:result is None 2023-06-27T12:37:14.442 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=ba3fff7216d65a7042e2ef64fe924764dc57a097 2023-06-27T12:37:14.587 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-27T12:37:14.610 INFO:teuthology.orchestra.run.smithi138.stdout:18.0.0-4614.gba3fff72.el8 2023-06-27T12:37:14.611 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4614.gba3fff72.el8 2023-06-27T12:37:14.612 INFO:teuthology.task.install:The correct ceph version 18.0.0-4614.gba3fff72 is installed. 2023-06-27T12:37:14.612 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=ba3fff7216d65a7042e2ef64fe924764dc57a097 2023-06-27T12:37:14.752 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-27T12:37:14.779 INFO:teuthology.orchestra.run.smithi159.stdout:18.0.0-4614.gba3fff72.el8 2023-06-27T12:37:14.780 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4614.gba3fff72.el8 2023-06-27T12:37:14.780 INFO:teuthology.task.install:The correct ceph version 18.0.0-4614.gba3fff72 is installed. 2023-06-27T12:37:14.781 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-27T12:37:14.781 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:14.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-27T12:37:14.809 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:14.809 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-27T12:37:14.853 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-27T12:37:14.854 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:14.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2023-06-27T12:37:14.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-27T12:37:14.947 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:14.947 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2023-06-27T12:37:14.977 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-27T12:37:15.046 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-27T12:37:15.047 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:15.047 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-27T12:37:15.073 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-27T12:37:15.140 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:15.140 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-27T12:37:15.170 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-27T12:37:15.238 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-27T12:37:15.238 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:15.239 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2023-06-27T12:37:15.265 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-27T12:37:15.332 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:15.333 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2023-06-27T12:37:15.361 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-27T12:37:15.430 INFO:teuthology.run_tasks:Running task ceph... 2023-06-27T12:37:15.505 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-27T12:37:15.506 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2023-06-27T12:37:15.509 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /var/log/ceph 2023-06-27T12:37:15.539 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-27T12:37:15.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-27T12:37:15.576 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-27T12:37:15.604 INFO:tasks.ceph:Creating extra log directories... 2023-06-27T12:37:15.605 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-27T12:37:15.643 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-27T12:37:15.692 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-27T12:37:15.693 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson seastar smp': 3, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-27T12:37:15.693 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2023-06-27_10:34:13-crimson-rados-wip-matanb-crimson-61651-distro-crimson-smithi/7317795', 'branch': 'wip-matanb-crimson-61651', 'description': 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-localized}', 'email': None, 'first_in_suite': False, 'job_id': '7317795', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2023-06-27_10:34:13-crimson-rados-wip-matanb-crimson-61651-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-61651'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson seastar smp': 3, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097', 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'fd62a8dfcb345fabed3eea6b3a461f1ca848a239'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'fd62a8dfcb345fabed3eea6b3a461f1ca848a239', 'targets': {'smithi138.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgThrJDXz65+FadHewTpcJkFYPfdgbxe7nQSh+DEDbi4mDOvkqyFOc3bi6SDKEqmTapP1If/P6toZyfU4STTzvBqlhh2il/4OROft9mllriUT9jcBaoO5km8c+uqV1q4mKRNhV1zEBE8lKageTV0j9OD54nnStOMHYOpxHhWBVmWnb/OuatqDFxHgmiLPsPkVCx1TU8QPDjEP9yzmK92xOVQ53XpvohTtuuhVl998uflK4iCbSNoFsMawVl2m34XbbQwSAm0Ar0hvUTK5Oa8LAJ81D8ChQjUaMzqc/r0/Me+iB1ScMpLdBeCOrcSGYE7Gt1xBqJ09bA21WVYxSdtmjXHiOZ0AaXnh5dYnz1U5DNgFsZgZsBLrukhDLXNWmzfZuW6K/d1KTMZtKBiqwrpFEwpYto7RyOQRY4VDsm+NrufI+Zt/hXJXW4UFLWT/0HcO5+AHk1IC4WI+zCGKEgw6EfpVo2CJYGPUGsiqR8yPJGye4GSgys4glyvdIwLAPcXE=', 'smithi159.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJT0rSA4ew9YllXsd9mCkiTOfvJut+5MH8gcgMh43A7RxIsTObiUnvZcaZH0xL72eEuprrxVs6iVsDdlSby1cY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson seastar smp': 3, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097', 'wait-for-scrub': False, 'cluster': 'ceph'}}, {'thrashosds': {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400}}, {'rados': {'clients': ['client.0'], 'localize_reads': True, 'objects': 50, 'op_weights': {'copy_from': 0, 'delete': 50, 'read': 100, 'rollback': 0, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3fdf1f2d90b4263fafe8c0792229f37d37a2ab95', 'timestamp': '2023-06-27_10:34:13', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-27T12:37:15.694 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-27T12:37:15.722 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-27T12:37:15.742 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2023-06-27T12:37:15.781 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m0777 -- /var/run/ceph 2023-06-27T12:37:15.808 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:15.809 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2023-06-27T12:37:15.864 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-27T12:37:15.864 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2023-06-27T12:37:15.921 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-27T12:37:15.921 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-27T12:37:15.922 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 410157 Links: 1 2023-06-27T12:37:15.922 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-27T12:37:15.922 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2023-06-27T12:37:15.922 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2023-06-27 12:37:09.546824222 +0000 2023-06-27T12:37:15.922 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2023-06-27 12:20:30.973542664 +0000 2023-06-27T12:37:15.922 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2023-06-27 12:20:30.973542664 +0000 2023-06-27T12:37:15.922 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2023-06-27T12:37:15.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-27T12:37:15.987 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2023-06-27T12:37:15.987 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2023-06-27T12:37:15.988 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000221031 s, 2.3 MB/s 2023-06-27T12:37:15.988 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-27T12:37:16.047 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2023-06-27T12:37:16.105 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-27T12:37:16.106 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-27T12:37:16.106 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 408343 Links: 1 2023-06-27T12:37:16.106 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-27T12:37:16.106 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2023-06-27T12:37:16.106 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2023-06-27 12:37:09.547824200 +0000 2023-06-27T12:37:16.107 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2023-06-27 12:20:31.493531468 +0000 2023-06-27T12:37:16.107 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2023-06-27 12:20:31.493531468 +0000 2023-06-27T12:37:16.107 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2023-06-27T12:37:16.107 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-27T12:37:16.171 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2023-06-27T12:37:16.171 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2023-06-27T12:37:16.171 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000216907 s, 2.4 MB/s 2023-06-27T12:37:16.172 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-27T12:37:16.231 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2023-06-27T12:37:16.289 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-27T12:37:16.289 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-27T12:37:16.289 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 408393 Links: 1 2023-06-27T12:37:16.289 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-27T12:37:16.290 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2023-06-27T12:37:16.290 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2023-06-27 12:37:09.547824200 +0000 2023-06-27T12:37:16.290 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2023-06-27 12:20:32.024520034 +0000 2023-06-27T12:37:16.290 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2023-06-27 12:20:32.024520034 +0000 2023-06-27T12:37:16.290 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2023-06-27T12:37:16.290 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-27T12:37:16.355 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2023-06-27T12:37:16.355 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2023-06-27T12:37:16.355 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000218094 s, 2.3 MB/s 2023-06-27T12:37:16.356 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-27T12:37:16.415 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2023-06-27T12:37:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-27T12:37:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-27T12:37:16.473 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 412049 Links: 1 2023-06-27T12:37:16.473 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-27T12:37:16.473 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2023-06-27T12:37:16.474 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2023-06-27 12:37:09.547824200 +0000 2023-06-27T12:37:16.474 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2023-06-27 12:20:32.544508838 +0000 2023-06-27T12:37:16.474 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2023-06-27 12:20:32.544508838 +0000 2023-06-27T12:37:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2023-06-27T12:37:16.474 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-27T12:37:16.538 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2023-06-27T12:37:16.538 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2023-06-27T12:37:16.539 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000207142 s, 2.5 MB/s 2023-06-27T12:37:16.539 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-27T12:37:16.598 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-06-27T12:37:16.598 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:16.599 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2023-06-27T12:37:16.618 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-27T12:37:16.619 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2023-06-27T12:37:16.678 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-27T12:37:16.678 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-27T12:37:16.678 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 400263 Links: 1 2023-06-27T12:37:16.679 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-27T12:37:16.679 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-06-27T12:37:16.679 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-06-27 12:35:34.389140891 +0000 2023-06-27T12:37:16.679 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-06-27 12:21:21.782035543 +0000 2023-06-27T12:37:16.680 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-06-27 12:21:21.782035543 +0000 2023-06-27T12:37:16.680 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-06-27T12:37:16.680 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-27T12:37:16.745 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-06-27T12:37:16.746 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-06-27T12:37:16.746 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000320297 s, 1.6 MB/s 2023-06-27T12:37:16.747 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-27T12:37:16.806 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2023-06-27T12:37:16.864 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-27T12:37:16.865 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-27T12:37:16.865 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 400302 Links: 1 2023-06-27T12:37:16.865 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-27T12:37:16.865 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-06-27T12:37:16.865 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-06-27 12:35:34.389140891 +0000 2023-06-27T12:37:16.865 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-06-27 12:21:22.299025825 +0000 2023-06-27T12:37:16.866 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-06-27 12:21:22.299025825 +0000 2023-06-27T12:37:16.866 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-06-27T12:37:16.866 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-27T12:37:16.930 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-06-27T12:37:16.931 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-06-27T12:37:16.931 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000236602 s, 2.2 MB/s 2023-06-27T12:37:16.932 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-27T12:37:16.991 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2023-06-27T12:37:17.049 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-27T12:37:17.049 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-27T12:37:17.050 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 406277 Links: 1 2023-06-27T12:37:17.050 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-27T12:37:17.050 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-06-27T12:37:17.050 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-06-27 12:35:34.389140891 +0000 2023-06-27T12:37:17.050 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-06-27 12:21:22.829015863 +0000 2023-06-27T12:37:17.050 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-06-27 12:21:22.829015863 +0000 2023-06-27T12:37:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-06-27T12:37:17.051 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-27T12:37:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-06-27T12:37:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-06-27T12:37:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.00021577 s, 2.4 MB/s 2023-06-27T12:37:17.116 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-27T12:37:17.175 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2023-06-27T12:37:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-27T12:37:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-27T12:37:17.234 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 413391 Links: 1 2023-06-27T12:37:17.234 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-27T12:37:17.234 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-06-27T12:37:17.234 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-06-27 12:35:34.389140891 +0000 2023-06-27T12:37:17.235 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-06-27 12:21:23.360005882 +0000 2023-06-27T12:37:17.235 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-06-27 12:21:23.360005882 +0000 2023-06-27T12:37:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-06-27T12:37:17.235 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-27T12:37:17.299 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-06-27T12:37:17.300 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-06-27T12:37:17.300 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000233498 s, 2.2 MB/s 2023-06-27T12:37:17.301 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-27T12:37:17.360 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-06-27T12:37:17.360 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi159.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2023-06-27T12:37:17.361 INFO:tasks.ceph:Generating config... 2023-06-27T12:37:17.362 INFO:tasks.ceph:[osd] debug monc = 20 2023-06-27T12:37:17.362 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-27T12:37:17.362 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2023-06-27T12:37:17.363 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-06-27T12:37:17.363 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-27T12:37:17.363 INFO:tasks.ceph:[osd] bluestore rocksdb cf = False 2023-06-27T12:37:17.363 INFO:tasks.ceph:[osd] crimson seastar smp = 3 2023-06-27T12:37:17.363 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-06-27T12:37:17.364 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-06-27T12:37:17.364 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-27T12:37:17.364 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-27T12:37:17.364 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-06-27T12:37:17.364 INFO:tasks.ceph:[osd] err to stderr = True 2023-06-27T12:37:17.365 INFO:tasks.ceph:[osd] log flush on exit = True 2023-06-27T12:37:17.365 INFO:tasks.ceph:[osd] log to file = False 2023-06-27T12:37:17.365 INFO:tasks.ceph:[osd] log to stderr = True 2023-06-27T12:37:17.365 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-27T12:37:17.365 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-27T12:37:17.366 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-27T12:37:17.366 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-06-27T12:37:17.366 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2023-06-27T12:37:17.366 INFO:tasks.ceph:[osd] osd delete sleep = 1 2023-06-27T12:37:17.366 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-27T12:37:17.367 INFO:tasks.ceph:[osd] osd max backfills = 3 2023-06-27T12:37:17.367 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-06-27T12:37:17.367 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-27T12:37:17.367 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-06-27T12:37:17.367 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-06-27T12:37:17.367 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-06-27T12:37:17.368 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2023-06-27T12:37:17.368 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2023-06-27T12:37:17.368 INFO:tasks.ceph:[global] ms client mode = crc 2023-06-27T12:37:17.368 INFO:tasks.ceph:[global] ms cluster mode = crc 2023-06-27T12:37:17.369 INFO:tasks.ceph:[global] ms mon client mode = crc 2023-06-27T12:37:17.369 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2023-06-27T12:37:17.369 INFO:tasks.ceph:[global] ms mon service mode = crc 2023-06-27T12:37:17.369 INFO:tasks.ceph:[global] ms service mode = crc 2023-06-27T12:37:17.369 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-06-27T12:37:17.369 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2023-06-27T12:37:17.370 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-27T12:37:17.370 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-27T12:37:17.370 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-27T12:37:17.370 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-27T12:37:17.370 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-27T12:37:17.371 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-06-27T12:37:17.371 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-06-27T12:37:17.371 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-06-27T12:37:17.371 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-06-27T12:37:17.371 INFO:tasks.ceph:[mon] osd pool default crimson = True 2023-06-27T12:37:17.372 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-06-27T12:37:17.372 INFO:tasks.ceph:Setting up mon.a... 2023-06-27T12:37:17.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-27T12:37:17.435 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.keyring 2023-06-27T12:37:17.438 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-27T12:37:17.492 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-27T12:37:17.519 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.138')] 2023-06-27T12:37:17.520 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.138', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson seastar smp': 3, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2023-06-27T12:37:17.521 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:17.522 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-27T12:37:17.577 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.138 --print /home/ubuntu/cephtest/ceph.monmap 2023-06-27T12:37:17.668 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:17.664+0000 7f256b765080 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:17.668 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:17.664+0000 7f256b765080 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:17.668 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:17.664+0000 7f256b765080 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:17.669 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-27T12:37:17.669 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: generated fsid 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9 2023-06-27T12:37:17.669 INFO:teuthology.orchestra.run.smithi138.stdout:setting min_mon_release = pacific 2023-06-27T12:37:17.669 INFO:teuthology.orchestra.run.smithi138.stdout:epoch 0 2023-06-27T12:37:17.669 INFO:teuthology.orchestra.run.smithi138.stdout:fsid 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9 2023-06-27T12:37:17.669 INFO:teuthology.orchestra.run.smithi138.stdout:last_changed 2023-06-27T12:37:17.665763+0000 2023-06-27T12:37:17.670 INFO:teuthology.orchestra.run.smithi138.stdout:created 2023-06-27T12:37:17.665763+0000 2023-06-27T12:37:17.670 INFO:teuthology.orchestra.run.smithi138.stdout:min_mon_release 16 (pacific) 2023-06-27T12:37:17.670 INFO:teuthology.orchestra.run.smithi138.stdout:election_strategy: 1 2023-06-27T12:37:17.670 INFO:teuthology.orchestra.run.smithi138.stdout:0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2023-06-27T12:37:17.670 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-06-27T12:37:17.671 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-27T12:37:17.689 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9... 2023-06-27T12:37:17.690 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-27T12:37:17.730 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-27T12:37:17.773 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2023-06-27T12:37:17.773 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2023-06-27T12:37:17.773 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-27T12:37:17.773 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2023-06-27T12:37:17.773 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.774 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2023-06-27T12:37:17.774 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.774 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2023-06-27T12:37:17.774 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.774 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2023-06-27T12:37:17.774 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2023-06-27T12:37:17.774 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.775 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2023-06-27T12:37:17.775 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2023-06-27T12:37:17.775 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.775 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-27T12:37:17.775 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2023-06-27T12:37:17.775 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.775 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2023-06-27T12:37:17.775 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-27T12:37:17.776 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.776 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2023-06-27T12:37:17.776 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.776 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2023-06-27T12:37:17.776 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2023-06-27T12:37:17.776 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2023-06-27T12:37:17.776 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2023-06-27T12:37:17.776 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2023-06-27T12:37:17.777 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2023-06-27T12:37:17.777 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2023-06-27T12:37:17.777 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-27T12:37:17.777 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-27T12:37:17.777 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2023-06-27T12:37:17.777 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.777 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-27T12:37:17.778 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.778 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2023-06-27T12:37:17.778 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.778 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2023-06-27T12:37:17.778 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.778 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2023-06-27T12:37:17.778 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2023-06-27T12:37:17.778 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2023-06-27T12:37:17.779 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = 172.21.15.138 2023-06-27T12:37:17.779 INFO:teuthology.orchestra.run.smithi138.stdout: enable experimental unrecoverable data corrupting features = crimson 2023-06-27T12:37:17.779 INFO:teuthology.orchestra.run.smithi138.stdout: ms client mode = crc 2023-06-27T12:37:17.779 INFO:teuthology.orchestra.run.smithi138.stdout: ms cluster mode = crc 2023-06-27T12:37:17.779 INFO:teuthology.orchestra.run.smithi138.stdout: ms mon client mode = crc 2023-06-27T12:37:17.779 INFO:teuthology.orchestra.run.smithi138.stdout: ms mon cluster mode = crc 2023-06-27T12:37:17.779 INFO:teuthology.orchestra.run.smithi138.stdout: ms mon service mode = crc 2023-06-27T12:37:17.779 INFO:teuthology.orchestra.run.smithi138.stdout: ms service mode = crc 2023-06-27T12:37:17.780 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_min_size = 2 2023-06-27T12:37:17.780 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_size = 3 2023-06-27T12:37:17.780 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9 2023-06-27T12:37:17.780 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.780 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2023-06-27T12:37:17.780 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2023-06-27T12:37:17.780 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.780 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2023-06-27T12:37:17.781 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 120 2023-06-27T12:37:17.781 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2023-06-27T12:37:17.781 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.781 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2023-06-27T12:37:17.781 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2023-06-27T12:37:17.781 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.781 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2023-06-27T12:37:17.781 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2023-06-27T12:37:17.782 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2023-06-27T12:37:17.782 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.782 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2023-06-27T12:37:17.782 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2023-06-27T12:37:17.782 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.782 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2023-06-27T12:37:17.782 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.782 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2023-06-27T12:37:17.783 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.783 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2023-06-27T12:37:17.783 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.783 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2023-06-27T12:37:17.783 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2023-06-27T12:37:17.783 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.783 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2023-06-27T12:37:17.783 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2023-06-27T12:37:17.784 INFO:teuthology.orchestra.run.smithi138.stdout: debug monc = 20 2023-06-27T12:37:17.784 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2023-06-27T12:37:17.784 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore compression algorithm = snappy 2023-06-27T12:37:17.784 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore compression mode = aggressive 2023-06-27T12:37:17.784 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2023-06-27T12:37:17.784 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore rocksdb cf = False 2023-06-27T12:37:17.784 INFO:teuthology.orchestra.run.smithi138.stdout: crimson seastar smp = 3 2023-06-27T12:37:17.784 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2023-06-27T12:37:17.785 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2023-06-27T12:37:17.785 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2023-06-27T12:37:17.785 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2023-06-27T12:37:17.785 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2023-06-27T12:37:17.785 INFO:teuthology.orchestra.run.smithi138.stdout: err to stderr = True 2023-06-27T12:37:17.785 INFO:teuthology.orchestra.run.smithi138.stdout: log flush on exit = True 2023-06-27T12:37:17.785 INFO:teuthology.orchestra.run.smithi138.stdout: log to file = False 2023-06-27T12:37:17.785 INFO:teuthology.orchestra.run.smithi138.stdout: log to stderr = True 2023-06-27T12:37:17.786 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2023-06-27T12:37:17.786 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2023-06-27T12:37:17.786 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2023-06-27T12:37:17.786 INFO:teuthology.orchestra.run.smithi138.stdout: osd blocked scrub grace period = 3600 2023-06-27T12:37:17.786 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug reject backfill probability = 0.3 2023-06-27T12:37:17.786 INFO:teuthology.orchestra.run.smithi138.stdout: osd delete sleep = 1 2023-06-27T12:37:17.786 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2023-06-27T12:37:17.786 INFO:teuthology.orchestra.run.smithi138.stdout: osd max backfills = 3 2023-06-27T12:37:17.787 INFO:teuthology.orchestra.run.smithi138.stdout: osd max markdown count = 1000 2023-06-27T12:37:17.787 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2023-06-27T12:37:17.787 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub min interval = 60 2023-06-27T12:37:17.787 INFO:teuthology.orchestra.run.smithi138.stdout: osd shutdown pgref assert = True 2023-06-27T12:37:17.787 INFO:teuthology.orchestra.run.smithi138.stdout: osd snap trim sleep = 2 2023-06-27T12:37:17.787 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.787 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2023-06-27T12:37:17.787 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2023-06-27T12:37:17.788 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2023-06-27T12:37:17.788 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2023-06-27T12:37:17.788 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2023-06-27T12:37:17.788 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2023-06-27T12:37:17.788 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2023-06-27T12:37:17.788 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2023-06-27T12:37:17.788 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.789 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2023-06-27T12:37:17.789 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2023-06-27T12:37:17.789 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2023-06-27T12:37:17.789 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2023-06-27T12:37:17.789 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2023-06-27T12:37:17.789 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2023-06-27T12:37:17.789 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2023-06-27T12:37:17.789 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2023-06-27T12:37:17.790 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2023-06-27T12:37:17.790 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2023-06-27T12:37:17.790 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2023-06-27T12:37:17.790 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.790 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-27T12:37:17.790 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2023-06-27T12:37:17.790 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2023-06-27T12:37:17.790 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.791 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2023-06-27T12:37:17.791 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-27T12:37:17.791 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-27T12:37:17.791 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.791 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2023-06-27T12:37:17.791 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2023-06-27T12:37:17.791 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.791 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2023-06-27T12:37:17.792 INFO:teuthology.orchestra.run.smithi138.stdout: mon min osdmap epochs = 50 2023-06-27T12:37:17.792 INFO:teuthology.orchestra.run.smithi138.stdout: mon osdmap full prune interval = 2 2023-06-27T12:37:17.792 INFO:teuthology.orchestra.run.smithi138.stdout: mon osdmap full prune min = 15 2023-06-27T12:37:17.792 INFO:teuthology.orchestra.run.smithi138.stdout: mon osdmap full prune txsize = 2 2023-06-27T12:37:17.792 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default crimson = True 2023-06-27T12:37:17.792 INFO:teuthology.orchestra.run.smithi138.stdout: paxos service trim min = 10 2023-06-27T12:37:17.792 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:17.792 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2023-06-27T12:37:17.793 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2023-06-27T12:37:17.793 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2023-06-27T12:37:17.793 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2023-06-27T12:37:17.793 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-27T12:37:17.793 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-27T12:37:17.793 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2023-06-27T12:37:17.794 INFO:teuthology.orchestra.run.smithi159.stdout:[global] 2023-06-27T12:37:17.795 INFO:teuthology.orchestra.run.smithi159.stdout: chdir = "" 2023-06-27T12:37:17.795 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-27T12:37:17.795 INFO:teuthology.orchestra.run.smithi159.stdout: auth supported = cephx 2023-06-27T12:37:17.795 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.795 INFO:teuthology.orchestra.run.smithi159.stdout: filestore xattr use omap = true 2023-06-27T12:37:17.795 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.795 INFO:teuthology.orchestra.run.smithi159.stdout: mon clock drift allowed = 1.000 2023-06-27T12:37:17.795 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.796 INFO:teuthology.orchestra.run.smithi159.stdout: osd crush chooseleaf type = 0 2023-06-27T12:37:17.796 INFO:teuthology.orchestra.run.smithi159.stdout: auth debug = true 2023-06-27T12:37:17.796 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.796 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on old message = true 2023-06-27T12:37:17.796 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on bug = true 2023-06-27T12:37:17.796 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.796 INFO:teuthology.orchestra.run.smithi159.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-27T12:37:17.797 INFO:teuthology.orchestra.run.smithi159.stdout: mon pg warn max object skew = 0 2023-06-27T12:37:17.797 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.797 INFO:teuthology.orchestra.run.smithi159.stdout: # disable pg_autoscaler by default for new pools 2023-06-27T12:37:17.797 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-27T12:37:17.797 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.797 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default size = 2 2023-06-27T12:37:17.797 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.797 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow primary affinity = true 2023-06-27T12:37:17.798 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow pg remap = true 2023-06-27T12:37:17.798 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on legacy crush tunables = false 2023-06-27T12:37:17.798 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on crush straw calc version zero = false 2023-06-27T12:37:17.798 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on no sortbitwise = false 2023-06-27T12:37:17.798 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on osd down out interval zero = false 2023-06-27T12:37:17.798 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on too few osds = false 2023-06-27T12:37:17.798 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-27T12:37:17.798 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-27T12:37:17.799 INFO:teuthology.orchestra.run.smithi159.stdout: mon_allow_pool_size_one = true 2023-06-27T12:37:17.799 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.799 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-27T12:37:17.799 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.799 INFO:teuthology.orchestra.run.smithi159.stdout: osd default data pool replay window = 5 2023-06-27T12:37:17.799 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.799 INFO:teuthology.orchestra.run.smithi159.stdout: mon allow pool delete = true 2023-06-27T12:37:17.799 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.800 INFO:teuthology.orchestra.run.smithi159.stdout: mon cluster log file level = debug 2023-06-27T12:37:17.800 INFO:teuthology.orchestra.run.smithi159.stdout: debug asserts on shutdown = true 2023-06-27T12:37:17.800 INFO:teuthology.orchestra.run.smithi159.stdout: mon health detail to clog = false 2023-06-27T12:37:17.800 INFO:teuthology.orchestra.run.smithi159.stdout: mon host = 172.21.15.138 2023-06-27T12:37:17.800 INFO:teuthology.orchestra.run.smithi159.stdout: enable experimental unrecoverable data corrupting features = crimson 2023-06-27T12:37:17.800 INFO:teuthology.orchestra.run.smithi159.stdout: ms client mode = crc 2023-06-27T12:37:17.800 INFO:teuthology.orchestra.run.smithi159.stdout: ms cluster mode = crc 2023-06-27T12:37:17.800 INFO:teuthology.orchestra.run.smithi159.stdout: ms mon client mode = crc 2023-06-27T12:37:17.801 INFO:teuthology.orchestra.run.smithi159.stdout: ms mon cluster mode = crc 2023-06-27T12:37:17.801 INFO:teuthology.orchestra.run.smithi159.stdout: ms mon service mode = crc 2023-06-27T12:37:17.801 INFO:teuthology.orchestra.run.smithi159.stdout: ms service mode = crc 2023-06-27T12:37:17.801 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_min_size = 2 2023-06-27T12:37:17.801 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_size = 3 2023-06-27T12:37:17.801 INFO:teuthology.orchestra.run.smithi159.stdout: fsid = 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9 2023-06-27T12:37:17.801 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.801 INFO:teuthology.orchestra.run.smithi159.stdout:[osd] 2023-06-27T12:37:17.802 INFO:teuthology.orchestra.run.smithi159.stdout: osd journal size = 100 2023-06-27T12:37:17.802 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.802 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub load threshold = 5.0 2023-06-27T12:37:17.802 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub max interval = 120 2023-06-27T12:37:17.802 INFO:teuthology.orchestra.run.smithi159.stdout: osd mclock profile = high_recovery_ops 2023-06-27T12:37:17.802 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.802 INFO:teuthology.orchestra.run.smithi159.stdout: osd recover clone overlap = true 2023-06-27T12:37:17.802 INFO:teuthology.orchestra.run.smithi159.stdout: osd recovery max chunk = 1048576 2023-06-27T12:37:17.803 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.803 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug shutdown = true 2023-06-27T12:37:17.803 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug op order = true 2023-06-27T12:37:17.803 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug verify stray on activate = true 2023-06-27T12:37:17.803 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.803 INFO:teuthology.orchestra.run.smithi159.stdout: osd open classes on start = true 2023-06-27T12:37:17.803 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug pg log writeout = true 2023-06-27T12:37:17.803 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.804 INFO:teuthology.orchestra.run.smithi159.stdout: osd deep scrub update digest min age = 30 2023-06-27T12:37:17.804 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.804 INFO:teuthology.orchestra.run.smithi159.stdout: osd map max advance = 10 2023-06-27T12:37:17.804 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.804 INFO:teuthology.orchestra.run.smithi159.stdout: journal zero on create = true 2023-06-27T12:37:17.804 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.804 INFO:teuthology.orchestra.run.smithi159.stdout: filestore ondisk finisher threads = 3 2023-06-27T12:37:17.804 INFO:teuthology.orchestra.run.smithi159.stdout: filestore apply finisher threads = 3 2023-06-27T12:37:17.805 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.805 INFO:teuthology.orchestra.run.smithi159.stdout: bdev debug aio = true 2023-06-27T12:37:17.805 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug misdirected ops = true 2023-06-27T12:37:17.805 INFO:teuthology.orchestra.run.smithi159.stdout: debug monc = 20 2023-06-27T12:37:17.805 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore block size = 96636764160 2023-06-27T12:37:17.805 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore compression algorithm = snappy 2023-06-27T12:37:17.805 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore compression mode = aggressive 2023-06-27T12:37:17.805 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore fsck on mount = True 2023-06-27T12:37:17.806 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore rocksdb cf = False 2023-06-27T12:37:17.806 INFO:teuthology.orchestra.run.smithi159.stdout: crimson seastar smp = 3 2023-06-27T12:37:17.806 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluefs = 20 2023-06-27T12:37:17.806 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluestore = 20 2023-06-27T12:37:17.806 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2023-06-27T12:37:17.806 INFO:teuthology.orchestra.run.smithi159.stdout: debug osd = 20 2023-06-27T12:37:17.806 INFO:teuthology.orchestra.run.smithi159.stdout: debug rocksdb = 10 2023-06-27T12:37:17.806 INFO:teuthology.orchestra.run.smithi159.stdout: err to stderr = True 2023-06-27T12:37:17.807 INFO:teuthology.orchestra.run.smithi159.stdout: log flush on exit = True 2023-06-27T12:37:17.807 INFO:teuthology.orchestra.run.smithi159.stdout: log to file = False 2023-06-27T12:37:17.807 INFO:teuthology.orchestra.run.smithi159.stdout: log to stderr = True 2023-06-27T12:37:17.807 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd backfillfull_ratio = 0.85 2023-06-27T12:37:17.807 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd full ratio = 0.9 2023-06-27T12:37:17.807 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd nearfull ratio = 0.8 2023-06-27T12:37:17.807 INFO:teuthology.orchestra.run.smithi159.stdout: osd blocked scrub grace period = 3600 2023-06-27T12:37:17.807 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug reject backfill probability = 0.3 2023-06-27T12:37:17.808 INFO:teuthology.orchestra.run.smithi159.stdout: osd delete sleep = 1 2023-06-27T12:37:17.808 INFO:teuthology.orchestra.run.smithi159.stdout: osd failsafe full ratio = 0.95 2023-06-27T12:37:17.808 INFO:teuthology.orchestra.run.smithi159.stdout: osd max backfills = 3 2023-06-27T12:37:17.808 INFO:teuthology.orchestra.run.smithi159.stdout: osd max markdown count = 1000 2023-06-27T12:37:17.808 INFO:teuthology.orchestra.run.smithi159.stdout: osd objectstore = bluestore 2023-06-27T12:37:17.808 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub min interval = 60 2023-06-27T12:37:17.808 INFO:teuthology.orchestra.run.smithi159.stdout: osd shutdown pgref assert = True 2023-06-27T12:37:17.809 INFO:teuthology.orchestra.run.smithi159.stdout: osd snap trim sleep = 2 2023-06-27T12:37:17.809 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.809 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr] 2023-06-27T12:37:17.809 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2023-06-27T12:37:17.809 INFO:teuthology.orchestra.run.smithi159.stdout: debug mgr = 20 2023-06-27T12:37:17.809 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2023-06-27T12:37:17.809 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2023-06-27T12:37:17.809 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2023-06-27T12:37:17.810 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2023-06-27T12:37:17.810 INFO:teuthology.orchestra.run.smithi159.stdout: mgr/telemetry/nag = false 2023-06-27T12:37:17.810 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.810 INFO:teuthology.orchestra.run.smithi159.stdout:[mon] 2023-06-27T12:37:17.810 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2023-06-27T12:37:17.810 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2023-06-27T12:37:17.810 INFO:teuthology.orchestra.run.smithi159.stdout: debug paxos = 20 2023-06-27T12:37:17.810 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2023-06-27T12:37:17.811 INFO:teuthology.orchestra.run.smithi159.stdout: mon data avail warn = 5 2023-06-27T12:37:17.811 INFO:teuthology.orchestra.run.smithi159.stdout: mon mgr mkfs grace = 240 2023-06-27T12:37:17.811 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2023-06-27T12:37:17.811 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd reporter subtree level = osd 2023-06-27T12:37:17.811 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd prime pg temp = true 2023-06-27T12:37:17.811 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2023-06-27T12:37:17.811 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.811 INFO:teuthology.orchestra.run.smithi159.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-27T12:37:17.812 INFO:teuthology.orchestra.run.smithi159.stdout: auth mon ticket ttl = 660 # 11m 2023-06-27T12:37:17.812 INFO:teuthology.orchestra.run.smithi159.stdout: auth service ticket ttl = 240 # 4m 2023-06-27T12:37:17.812 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.812 INFO:teuthology.orchestra.run.smithi159.stdout: # don't complain about insecure global_id in the test suite 2023-06-27T12:37:17.812 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-27T12:37:17.812 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-27T12:37:17.812 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.812 INFO:teuthology.orchestra.run.smithi159.stdout: # 1m isn't quite enough 2023-06-27T12:37:17.813 INFO:teuthology.orchestra.run.smithi159.stdout: mon_down_mkfs_grace = 2m 2023-06-27T12:37:17.813 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.813 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_filestore_osds = false 2023-06-27T12:37:17.813 INFO:teuthology.orchestra.run.smithi159.stdout: mon min osdmap epochs = 50 2023-06-27T12:37:17.813 INFO:teuthology.orchestra.run.smithi159.stdout: mon osdmap full prune interval = 2 2023-06-27T12:37:17.813 INFO:teuthology.orchestra.run.smithi159.stdout: mon osdmap full prune min = 15 2023-06-27T12:37:17.813 INFO:teuthology.orchestra.run.smithi159.stdout: mon osdmap full prune txsize = 2 2023-06-27T12:37:17.813 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default crimson = True 2023-06-27T12:37:17.814 INFO:teuthology.orchestra.run.smithi159.stdout: paxos service trim min = 10 2023-06-27T12:37:17.814 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-06-27T12:37:17.814 INFO:teuthology.orchestra.run.smithi159.stdout:[client] 2023-06-27T12:37:17.814 INFO:teuthology.orchestra.run.smithi159.stdout: rgw cache enabled = true 2023-06-27T12:37:17.814 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable ops log = true 2023-06-27T12:37:17.814 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable usage log = true 2023-06-27T12:37:17.814 INFO:teuthology.orchestra.run.smithi159.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-27T12:37:17.814 INFO:teuthology.orchestra.run.smithi159.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-27T12:37:17.815 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.a] 2023-06-27T12:37:17.815 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-27T12:37:17.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-27T12:37:17.899 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-27T12:37:17.899 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:17.900 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-27T12:37:17.917 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:17.917 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-27T12:37:17.974 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2023-06-27T12:37:17.975 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:17.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-27T12:37:17.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-27T12:37:18.050 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:18.051 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-27T12:37:18.108 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi159.front.sepia.ceph.com 2023-06-27T12:37:18.108 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:18.108 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-27T12:37:18.109 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-27T12:37:18.145 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:18.145 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-27T12:37:18.203 INFO:tasks.ceph:Setting up mon nodes... 2023-06-27T12:37:18.203 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-27T12:37:18.204 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-27T12:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-27T12:37:18.316 INFO:tasks.ceph:Setting up mds nodes... 2023-06-27T12:37:18.316 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-27T12:37:18.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-27T12:37:18.367 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-27T12:37:18.380 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-27T12:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-06-27T12:37:18.443 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-27T12:37:18.443 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi159.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2023-06-27T12:37:18.444 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-27T12:37:18.470 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-06-27T12:37:18.471 INFO:tasks.ceph:role: osd.0 2023-06-27T12:37:18.471 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2023-06-27T12:37:18.471 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-27T12:37:18.545 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-27T12:37:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-27T12:37:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-27T12:37:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-27T12:37:18.545 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-27T12:37:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2023-06-27T12:37:18.546 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-27T12:37:18.546 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-27T12:37:18.546 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-27T12:37:18.546 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-27T12:37:18.977 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2023-06-27T12:37:18.978 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2023-06-27T12:37:18.979 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-27T12:37:19.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-27T12:37:19.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-27T12:37:19.239 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-06-27T12:37:19.240 INFO:tasks.ceph:role: osd.1 2023-06-27T12:37:19.240 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2023-06-27T12:37:19.240 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-27T12:37:19.314 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-27T12:37:19.315 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-27T12:37:19.315 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-27T12:37:19.315 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-27T12:37:19.315 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-27T12:37:19.316 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2023-06-27T12:37:19.316 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-27T12:37:19.316 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-27T12:37:19.317 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-27T12:37:19.317 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-27T12:37:19.742 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2023-06-27T12:37:19.744 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2023-06-27T12:37:19.744 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-27T12:37:19.853 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-27T12:37:19.884 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-27T12:37:20.701 INFO:teuthology.orchestra.run.smithi138.stderr:WARNING: debug mode. Not for benchmarking or production 2023-06-27T12:37:20.702 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:20,700 seastar - Reactor backend: epoll 2023-06-27T12:37:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:20,707 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-06-27T12:37:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:20,707 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-06-27T12:37:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:20,707 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-06-27T12:37:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:20,707 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-06-27T12:37:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:==135716==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-06-27T12:37:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:==135716==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe37de4000; bottom 0x7f6fbd5f3000; size: 0x008e7a7f1000 (611940503552) 2023-06-27T12:37:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:False positive error reports may follow 2023-06-27T12:37:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-06-27T12:37:20.755 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:20,753 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-06-27T12:37:20.755 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:20,753 [shard 0] osd - get_early_config: set --smp 3 2023-06-27T12:37:21.323 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2023-06-27T12:37:21.324 INFO:teuthology.orchestra.run.smithi138.stderr:Suppressions used: 2023-06-27T12:37:21.324 INFO:teuthology.orchestra.run.smithi138.stderr: count bytes template 2023-06-27T12:37:21.324 INFO:teuthology.orchestra.run.smithi138.stderr: 1 8 InitModule 2023-06-27T12:37:21.324 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2023-06-27T12:37:21.324 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:37:21.377 INFO:teuthology.orchestra.run.smithi138.stderr:WARNING: debug mode. Not for benchmarking or production 2023-06-27T12:37:21.377 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:21,375 seastar - Reactor backend: epoll 2023-06-27T12:37:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:21,377 [shard 2] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-06-27T12:37:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:21,377 [shard 2] seastar - IO queue uses 0.75ms latency goal for device 0 2023-06-27T12:37:21.380 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:21,377 [shard 2] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-06-27T12:37:21.380 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:21,377 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-06-27T12:37:21.382 INFO:teuthology.orchestra.run.smithi138.stderr:==135714==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-06-27T12:37:21.388 INFO:teuthology.orchestra.run.smithi138.stderr:==135714==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe37de4000; bottom 0x7f6fba4ec000; size: 0x008e7d8f8000 (611991912448) 2023-06-27T12:37:21.388 INFO:teuthology.orchestra.run.smithi138.stderr:False positive error reports may follow 2023-06-27T12:37:21.388 INFO:teuthology.orchestra.run.smithi138.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-06-27T12:37:25.493 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2023-06-27T12:37:25.493 INFO:teuthology.orchestra.run.smithi138.stderr:Suppressions used: 2023-06-27T12:37:25.493 INFO:teuthology.orchestra.run.smithi138.stderr: count bytes template 2023-06-27T12:37:25.493 INFO:teuthology.orchestra.run.smithi138.stderr: 1 8 InitModule 2023-06-27T12:37:25.493 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2023-06-27T12:37:25.493 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:37:25.494 INFO:teuthology.orchestra.run.smithi138.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9 2023-06-27T12:37:25.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-27T12:37:25.582 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-27T12:37:26.382 INFO:teuthology.orchestra.run.smithi138.stderr:WARNING: debug mode. Not for benchmarking or production 2023-06-27T12:37:26.383 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:26,380 seastar - Reactor backend: epoll 2023-06-27T12:37:26.395 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:26,392 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-06-27T12:37:26.395 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:26,392 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-06-27T12:37:26.395 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:26,392 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-06-27T12:37:26.395 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:26,392 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-06-27T12:37:26.397 INFO:teuthology.orchestra.run.smithi138.stderr:==135914==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-06-27T12:37:26.403 INFO:teuthology.orchestra.run.smithi138.stderr:==135914==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd02b97000; bottom 0x7fb20a197000; size: 0x004af8a00000 (321998815232) 2023-06-27T12:37:26.403 INFO:teuthology.orchestra.run.smithi138.stderr:False positive error reports may follow 2023-06-27T12:37:26.403 INFO:teuthology.orchestra.run.smithi138.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-06-27T12:37:26.440 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:26,437 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-06-27T12:37:26.440 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:26,437 [shard 0] osd - get_early_config: set --smp 3 2023-06-27T12:37:27.028 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2023-06-27T12:37:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:Suppressions used: 2023-06-27T12:37:27.029 INFO:teuthology.orchestra.run.smithi138.stderr: count bytes template 2023-06-27T12:37:27.029 INFO:teuthology.orchestra.run.smithi138.stderr: 1 8 InitModule 2023-06-27T12:37:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2023-06-27T12:37:27.029 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:37:27.086 INFO:teuthology.orchestra.run.smithi138.stderr:WARNING: debug mode. Not for benchmarking or production 2023-06-27T12:37:27.086 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:27,084 seastar - Reactor backend: epoll 2023-06-27T12:37:27.095 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:27,093 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-06-27T12:37:27.096 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:27,093 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2023-06-27T12:37:27.096 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:27,093 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-06-27T12:37:27.096 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2023-06-27 12:37:27,093 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-06-27T12:37:27.098 INFO:teuthology.orchestra.run.smithi138.stderr:==135912==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-06-27T12:37:27.104 INFO:teuthology.orchestra.run.smithi138.stderr:==135912==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd02b97000; bottom 0x7fb207090000; size: 0x004afbb07000 (322050224128) 2023-06-27T12:37:27.104 INFO:teuthology.orchestra.run.smithi138.stderr:False positive error reports may follow 2023-06-27T12:37:27.104 INFO:teuthology.orchestra.run.smithi138.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-06-27T12:37:31.164 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2023-06-27T12:37:31.164 INFO:teuthology.orchestra.run.smithi138.stderr:Suppressions used: 2023-06-27T12:37:31.164 INFO:teuthology.orchestra.run.smithi138.stderr: count bytes template 2023-06-27T12:37:31.164 INFO:teuthology.orchestra.run.smithi138.stderr: 1 8 InitModule 2023-06-27T12:37:31.165 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2023-06-27T12:37:31.165 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:37:31.165 INFO:teuthology.orchestra.run.smithi138.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9 2023-06-27T12:37:31.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-27T12:37:31.276 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-27T12:37:31.309 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-06-27T12:37:31.309 INFO:tasks.ceph:role: osd.2 2023-06-27T12:37:31.309 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi159.front.sepia.ceph.com 2023-06-27T12:37:31.310 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-27T12:37:31.388 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-27T12:37:31.388 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-27T12:37:31.388 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-27T12:37:31.389 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-27T12:37:31.389 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-27T12:37:31.389 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2023-06-27T12:37:31.389 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-27T12:37:31.390 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-27T12:37:31.390 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-27T12:37:31.390 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-27T12:37:31.817 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2023-06-27T12:37:31.819 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2023-06-27T12:37:31.819 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2023-06-27T12:37:31.998 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-27T12:37:32.038 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-27T12:37:32.109 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-06-27T12:37:32.110 INFO:tasks.ceph:role: osd.3 2023-06-27T12:37:32.110 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi159.front.sepia.ceph.com 2023-06-27T12:37:32.110 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-27T12:37:32.185 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-27T12:37:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-27T12:37:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-27T12:37:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-27T12:37:32.186 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-27T12:37:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2023-06-27T12:37:32.187 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-27T12:37:32.187 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-27T12:37:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-27T12:37:32.187 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-27T12:37:32.611 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2023-06-27T12:37:32.612 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2023-06-27T12:37:32.613 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2023-06-27T12:37:32.728 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-27T12:37:32.761 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-27T12:37:33.597 INFO:teuthology.orchestra.run.smithi159.stderr:WARNING: debug mode. Not for benchmarking or production 2023-06-27T12:37:33.598 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:33,595 seastar - Reactor backend: epoll 2023-06-27T12:37:33.606 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:33,603 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-06-27T12:37:33.606 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:33,603 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-06-27T12:37:33.606 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:33,603 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-06-27T12:37:33.606 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:33,603 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-06-27T12:37:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:==135550==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-06-27T12:37:33.614 INFO:teuthology.orchestra.run.smithi159.stderr:==135550==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd8cebc000; bottom 0x7fe51f68e000; size: 0x00186d82e000 (104916508672) 2023-06-27T12:37:33.614 INFO:teuthology.orchestra.run.smithi159.stderr:False positive error reports may follow 2023-06-27T12:37:33.615 INFO:teuthology.orchestra.run.smithi159.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-06-27T12:37:33.651 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:33,648 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-06-27T12:37:33.651 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:33,648 [shard 0] osd - get_early_config: set --smp 3 2023-06-27T12:37:34.221 INFO:teuthology.orchestra.run.smithi159.stderr:----------------------------------------------------- 2023-06-27T12:37:34.221 INFO:teuthology.orchestra.run.smithi159.stderr:Suppressions used: 2023-06-27T12:37:34.221 INFO:teuthology.orchestra.run.smithi159.stderr: count bytes template 2023-06-27T12:37:34.221 INFO:teuthology.orchestra.run.smithi159.stderr: 1 8 InitModule 2023-06-27T12:37:34.222 INFO:teuthology.orchestra.run.smithi159.stderr:----------------------------------------------------- 2023-06-27T12:37:34.222 INFO:teuthology.orchestra.run.smithi159.stderr: 2023-06-27T12:37:34.275 INFO:teuthology.orchestra.run.smithi159.stderr:WARNING: debug mode. Not for benchmarking or production 2023-06-27T12:37:34.276 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:34,273 seastar - Reactor backend: epoll 2023-06-27T12:37:34.284 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:34,281 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-06-27T12:37:34.285 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:34,281 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-06-27T12:37:34.285 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:34,282 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-06-27T12:37:34.285 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:34,282 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-06-27T12:37:34.287 INFO:teuthology.orchestra.run.smithi159.stderr:==135548==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-06-27T12:37:34.293 INFO:teuthology.orchestra.run.smithi159.stderr:==135548==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd8cebc000; bottom 0x7fe51c587000; size: 0x001870935000 (104967917568) 2023-06-27T12:37:34.294 INFO:teuthology.orchestra.run.smithi159.stderr:False positive error reports may follow 2023-06-27T12:37:34.294 INFO:teuthology.orchestra.run.smithi159.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-06-27T12:37:38.410 INFO:teuthology.orchestra.run.smithi159.stderr:----------------------------------------------------- 2023-06-27T12:37:38.410 INFO:teuthology.orchestra.run.smithi159.stderr:Suppressions used: 2023-06-27T12:37:38.411 INFO:teuthology.orchestra.run.smithi159.stderr: count bytes template 2023-06-27T12:37:38.411 INFO:teuthology.orchestra.run.smithi159.stderr: 1 8 InitModule 2023-06-27T12:37:38.411 INFO:teuthology.orchestra.run.smithi159.stderr:----------------------------------------------------- 2023-06-27T12:37:38.411 INFO:teuthology.orchestra.run.smithi159.stderr: 2023-06-27T12:37:38.412 INFO:teuthology.orchestra.run.smithi159.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9 2023-06-27T12:37:38.469 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-27T12:37:38.497 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-27T12:37:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:WARNING: debug mode. Not for benchmarking or production 2023-06-27T12:37:39.301 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,298 seastar - Reactor backend: epoll 2023-06-27T12:37:39.309 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,306 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-06-27T12:37:39.309 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,306 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-06-27T12:37:39.309 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,306 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-06-27T12:37:39.309 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,306 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-06-27T12:37:39.311 INFO:teuthology.orchestra.run.smithi159.stderr:==135749==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-06-27T12:37:39.317 INFO:teuthology.orchestra.run.smithi159.stderr:==135749==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd4ebd6000; bottom 0x7fa6936dd000; size: 0x0056bb4f9000 (372509741056) 2023-06-27T12:37:39.317 INFO:teuthology.orchestra.run.smithi159.stderr:False positive error reports may follow 2023-06-27T12:37:39.318 INFO:teuthology.orchestra.run.smithi159.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-06-27T12:37:39.354 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,351 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-06-27T12:37:39.354 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,351 [shard 0] osd - get_early_config: set --smp 3 2023-06-27T12:37:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:----------------------------------------------------- 2023-06-27T12:37:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:Suppressions used: 2023-06-27T12:37:39.917 INFO:teuthology.orchestra.run.smithi159.stderr: count bytes template 2023-06-27T12:37:39.917 INFO:teuthology.orchestra.run.smithi159.stderr: 1 8 InitModule 2023-06-27T12:37:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:----------------------------------------------------- 2023-06-27T12:37:39.917 INFO:teuthology.orchestra.run.smithi159.stderr: 2023-06-27T12:37:39.974 INFO:teuthology.orchestra.run.smithi159.stderr:WARNING: debug mode. Not for benchmarking or production 2023-06-27T12:37:39.975 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,972 seastar - Reactor backend: epoll 2023-06-27T12:37:39.976 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,974 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-06-27T12:37:39.977 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,974 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2023-06-27T12:37:39.977 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,974 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-06-27T12:37:39.977 INFO:teuthology.orchestra.run.smithi159.stderr:INFO 2023-06-27 12:37:39,974 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-06-27T12:37:39.979 INFO:teuthology.orchestra.run.smithi159.stderr:==135747==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-06-27T12:37:39.985 INFO:teuthology.orchestra.run.smithi159.stderr:==135747==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd4ebd6000; bottom 0x7fa6905d6000; size: 0x0056be600000 (372561149952) 2023-06-27T12:37:39.985 INFO:teuthology.orchestra.run.smithi159.stderr:False positive error reports may follow 2023-06-27T12:37:39.986 INFO:teuthology.orchestra.run.smithi159.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-06-27T12:37:44.034 INFO:teuthology.orchestra.run.smithi159.stderr:----------------------------------------------------- 2023-06-27T12:37:44.034 INFO:teuthology.orchestra.run.smithi159.stderr:Suppressions used: 2023-06-27T12:37:44.034 INFO:teuthology.orchestra.run.smithi159.stderr: count bytes template 2023-06-27T12:37:44.034 INFO:teuthology.orchestra.run.smithi159.stderr: 1 8 InitModule 2023-06-27T12:37:44.035 INFO:teuthology.orchestra.run.smithi159.stderr:----------------------------------------------------- 2023-06-27T12:37:44.035 INFO:teuthology.orchestra.run.smithi159.stderr: 2023-06-27T12:37:44.035 INFO:teuthology.orchestra.run.smithi159.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9 2023-06-27T12:37:44.118 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-27T12:37:44.145 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-27T12:37:44.146 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:44.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-27T12:37:44.178 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:44.179 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-27T12:37:44.249 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:44.250 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-27T12:37:44.277 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:44.278 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-27T12:37:44.344 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:44.345 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-27T12:37:44.411 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:44.411 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-27T12:37:44.432 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:44.432 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-06-27T12:37:44.470 INFO:tasks.ceph:Adding keys to all mons... 2023-06-27T12:37:44.471 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-27T12:37:44.502 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2023-06-27T12:37:44.502 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAB2JpkYdAYGhAA48EiruKEW85azhk1O697fQ== 2023-06-27T12:37:44.502 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2023-06-27T12:37:44.502 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAH2JpkVypTCRAA8gQVI9N1QXRrK6VqYEm/8A== 2023-06-27T12:37:44.502 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2023-06-27T12:37:44.503 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQD+15pk41GEEhAA50K5v6YLYWodrF642rsXLg== 2023-06-27T12:37:44.503 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2023-06-27T12:37:44.503 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAO2Jpki3htFBAAga0l/BKZ3ytsMHMS7zN/4A== 2023-06-27T12:37:44.503 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2023-06-27T12:37:44.503 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAU2JpkOsIZAhAA/f20qXGPfaF2Ey3bEMVJ0w== 2023-06-27T12:37:44.503 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2023-06-27T12:37:44.504 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQD+15pkVoq+FRAAnu4LyoYkiSFAj+TyjWgI0g== 2023-06-27T12:37:44.504 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2023-06-27T12:37:44.504 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQD+15pkGdaLGRAAuFcm2Ld3VX+W3ZcTQ5AbkA== 2023-06-27T12:37:44.505 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-27T12:37:44.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-27T12:37:44.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-27T12:37:44.737 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-27T12:37:44.797 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-27T12:37:44.858 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-27T12:37:44.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-27T12:37:44.987 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-27T12:37:44.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-27T12:37:45.014 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-27T12:37:45.115 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:45.111+0000 7f21c3875c40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.115 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:45.111+0000 7f21c3875c40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-27T12:37:45.159 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-27T12:37:45.217 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-27T12:37:45.217 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-27T12:37:45.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-27T12:37:45.260 INFO:tasks.ceph.mon.a:Started 2023-06-27T12:37:45.260 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-27T12:37:45.261 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-27T12:37:45.261 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-27T12:37:45.263 INFO:tasks.ceph.mgr.x:Started 2023-06-27T12:37:45.264 INFO:tasks.ceph:Setting crush tunables to default 2023-06-27T12:37:45.264 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd crush tunables default 2023-06-27T12:37:45.344 INFO:tasks.ceph.mon.a.smithi138.stderr:2023-06-27T12:37:45.340+0000 7f3f3fdedc40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.344 INFO:tasks.ceph.mon.a.smithi138.stderr:2023-06-27T12:37:45.340+0000 7f3f3fdedc40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.344 INFO:tasks.ceph.mon.a.smithi138.stderr:2023-06-27T12:37:45.340+0000 7f3f3fdedc40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.358 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:45.354+0000 7f449f4311c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.394 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:45.385+0000 7f4a9d42e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.409 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:45.405+0000 7f4a9d42e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.489 INFO:teuthology.orchestra.run.smithi138.stderr:adjusted tunables profile to default 2023-06-27T12:37:45.500 INFO:tasks.ceph:check_enable_crimson: True 2023-06-27T12:37:45.501 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2023-06-27T12:37:45.501 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2023-06-27T12:37:45.561 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:45.558+0000 7f449f4311c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.562 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:45.558+0000 7f449f4311c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.627 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:45.619+0000 7f9d3d0ca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.630 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:45.626+0000 7f9d3d0ca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:45.719 INFO:tasks.ceph.mon.a.smithi138.stderr:2023-06-27T12:37:45.715+0000 7f3f330a9700 -1 WARNING: experimental feature 'crimson' is enabled 2023-06-27T12:37:45.719 INFO:tasks.ceph.mon.a.smithi138.stderr:Please be aware that this feature is experimental, untested, 2023-06-27T12:37:45.719 INFO:tasks.ceph.mon.a.smithi138.stderr:unsupported, and may result in data corruption, data loss, 2023-06-27T12:37:45.720 INFO:tasks.ceph.mon.a.smithi138.stderr:and/or irreparable damage to your cluster. Do not use 2023-06-27T12:37:45.720 INFO:tasks.ceph.mon.a.smithi138.stderr:feature with important data. 2023-06-27T12:37:45.720 INFO:tasks.ceph.mon.a.smithi138.stderr: 2023-06-27T12:37:45.835 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:45.831+0000 7f449f4311c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-27T12:37:46.035 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:46.031+0000 7f449f4311c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-27T12:37:46.412 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:46.408+0000 7f449f4311c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-27T12:37:46.506 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-27T12:37:46.507 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:46.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-27T12:37:46.538 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2023-06-27T12:37:46.539 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-27T12:37:46.608 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:46.609 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-27T12:37:46.637 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-06-27T12:37:46.638 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-27T12:37:46.705 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --cluster ceph osd new 25bbe3c9-50fb-430c-8ae5-afc94081c458 0 2023-06-27T12:37:46.874 INFO:teuthology.orchestra.run.smithi159.stderr:2023-06-27T12:37:46.865+0000 7fe5cf3d5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:46.877 INFO:teuthology.orchestra.run.smithi159.stderr:2023-06-27T12:37:46.873+0000 7fe5cf3d5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:46.909 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:46.905+0000 7f449f4311c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi159.stdout:0 2023-06-27T12:37:46.982 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --cluster ceph osd new 3b06ea8c-fece-4d4b-bafe-f670227862c9 1 2023-06-27T12:37:47.112 INFO:teuthology.orchestra.run.smithi159.stderr:2023-06-27T12:37:47.103+0000 7f20b20c2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:47.115 INFO:teuthology.orchestra.run.smithi159.stderr:2023-06-27T12:37:47.111+0000 7f20b20c2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:47.115 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:47.112+0000 7f449f4311c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-27T12:37:47.206 INFO:teuthology.orchestra.run.smithi159.stdout:1 2023-06-27T12:37:47.218 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --cluster ceph osd new ac444bcf-4783-4337-9729-05a1b08d60bc 2 2023-06-27T12:37:47.348 INFO:teuthology.orchestra.run.smithi159.stderr:2023-06-27T12:37:47.339+0000 7f50178b7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:47.351 INFO:teuthology.orchestra.run.smithi159.stderr:2023-06-27T12:37:47.347+0000 7f50178b7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:47.443 INFO:teuthology.orchestra.run.smithi159.stdout:2 2023-06-27T12:37:47.457 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --cluster ceph osd new f3f5961d-51f1-4b83-9cb4-ce14355d9dd6 3 2023-06-27T12:37:47.494 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:47.491+0000 7f449f4311c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-27T12:37:47.587 INFO:teuthology.orchestra.run.smithi159.stderr:2023-06-27T12:37:47.577+0000 7fab9d9ba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:47.589 INFO:teuthology.orchestra.run.smithi159.stderr:2023-06-27T12:37:47.586+0000 7fab9d9ba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:47.680 INFO:teuthology.orchestra.run.smithi159.stdout:3 2023-06-27T12:37:47.693 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2023-06-27T12:37:47.731 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-27T12:37:47.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2023-06-27T12:37:47.774 INFO:tasks.ceph.osd.0:Started 2023-06-27T12:37:47.776 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2023-06-27T12:37:47.812 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-27T12:37:47.812 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2023-06-27T12:37:47.854 INFO:tasks.ceph.osd.1:Started 2023-06-27T12:37:47.854 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2023-06-27T12:37:47.886 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-27T12:37:47.887 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2023-06-27T12:37:47.928 INFO:tasks.ceph.osd.2:Started 2023-06-27T12:37:47.929 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2023-06-27T12:37:47.961 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-27T12:37:47.962 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2023-06-27T12:37:48.004 INFO:tasks.ceph.osd.3:Started 2023-06-27T12:37:48.005 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-27T12:37:48.158 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:48.150+0000 7f1fb7d6a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:48.162 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:48.158+0000 7f1fb7d6a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:48.216 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:48.212+0000 7f449f4311c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-27T12:37:48.240 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:48.240 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":7,"fsid":"0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9","created":"2023-06-27T12:37:45.394408+0000","modified":"2023-06-27T12:37:47.674294+0000","last_up_change":"0.000000","last_in_change":"2023-06-27T12:37:47.674294+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"25bbe3c9-50fb-430c-8ae5-afc94081c458","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3b06ea8c-fece-4d4b-bafe-f670227862c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ac444bcf-4783-4337-9729-05a1b08d60bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f3f5961d-51f1-4b83-9cb4-ce14355d9dd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-27T12:37:48.258 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-27T12:37:48.259 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-27T12:37:48.316 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:48.313+0000 7f449f4311c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-27T12:37:48.482 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:48.479+0000 7f449f4311c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-27T12:37:48.568 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:48.564+0000 7f449f4311c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-27T12:37:48.661 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-27T12:37:48.779 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:48.776+0000 7f449f4311c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-27T12:37:48.784 INFO:teuthology.misc.health.smithi138.stderr:2023-06-27T12:37:48.777+0000 7f4bca1e6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:48.787 INFO:teuthology.misc.health.smithi138.stderr:2023-06-27T12:37:48.783+0000 7f4bca1e6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:48.863 INFO:teuthology.misc.health.smithi138.stdout: 2023-06-27T12:37:48.863 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":7,"fsid":"0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9","created":"2023-06-27T12:37:45.394408+0000","modified":"2023-06-27T12:37:47.674294+0000","last_up_change":"0.000000","last_in_change":"2023-06-27T12:37:47.674294+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"25bbe3c9-50fb-430c-8ae5-afc94081c458","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3b06ea8c-fece-4d4b-bafe-f670227862c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ac444bcf-4783-4337-9729-05a1b08d60bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f3f5961d-51f1-4b83-9cb4-ce14355d9dd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-27T12:37:48.868 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:48.864+0000 7f449f4311c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-27T12:37:48.874 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-06-27T12:37:48.950 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:48.946+0000 7f449f4311c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-27T12:37:49.033 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:49.030+0000 7f449f4311c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-27T12:37:49.239 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:49.235+0000 7f449f4311c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-27T12:37:49.332 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:49.328+0000 7f449f4311c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-27T12:37:49.651 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:49.647+0000 7f449f4311c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-27T12:37:49.732 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:49.728+0000 7f449f4311c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-27T12:37:50.283 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:50.279+0000 7f449f4311c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-27T12:37:50.671 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:50.668+0000 7f449f4311c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-27T12:37:50.821 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:50.817+0000 7f449f4311c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-27T12:37:51.241 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:51.238+0000 7f449f4311c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-27T12:37:51.404 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:51.400+0000 7f449f4311c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-27T12:37:51.572 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:51.568+0000 7f449f4311c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-27T12:37:51.888 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:51.884+0000 7f449f4311c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-27T12:37:52.914 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:37:52.910+0000 7f446334d700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-06-27T12:37:55.278 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-27T12:37:55.409 INFO:teuthology.misc.health.smithi138.stderr:2023-06-27T12:37:55.402+0000 7f7ad93eb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:55.412 INFO:teuthology.misc.health.smithi138.stderr:2023-06-27T12:37:55.408+0000 7f7ad93eb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:55.606 INFO:teuthology.misc.health.smithi138.stdout: 2023-06-27T12:37:55.606 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":11,"fsid":"0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9","created":"2023-06-27T12:37:45.394408+0000","modified":"2023-06-27T12:37:54.794168+0000","last_up_change":"2023-06-27T12:37:53.788853+0000","last_in_change":"2023-06-27T12:37:47.674294+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-27T12:37:53.945979+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25bbe3c9-50fb-430c-8ae5-afc94081c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2316736315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":2316736315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6805","nonce":2316736315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2316736315}]},"public_addr":"172.21.15.138:6800/2316736315","cluster_addr":"172.21.15.138:6801/2316736315","heartbeat_back_addr":"172.21.15.138:6805/2316736315","heartbeat_front_addr":"172.21.15.138:6804/2316736315","state":["exists","up"]},{"osd":1,"uuid":"3b06ea8c-fece-4d4b-bafe-f670227862c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":2294192134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2294192134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2294192134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":2294192134}]},"public_addr":"172.21.15.138:6803/2294192134","cluster_addr":"172.21.15.138:6802/2294192134","heartbeat_back_addr":"172.21.15.138:6806/2294192134","heartbeat_front_addr":"172.21.15.138:6807/2294192134","state":["exists","up"]},{"osd":2,"uuid":"ac444bcf-4783-4337-9729-05a1b08d60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1657039289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6805","nonce":1657039289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6809","nonce":1657039289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1657039289}]},"public_addr":"172.21.15.159:6804/1657039289","cluster_addr":"172.21.15.159:6805/1657039289","heartbeat_back_addr":"172.21.15.159:6809/1657039289","heartbeat_front_addr":"172.21.15.159:6808/1657039289","state":["exists","up"]},{"osd":3,"uuid":"f3f5961d-51f1-4b83-9cb4-ce14355d9dd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2263671727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6803","nonce":2263671727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2263671727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6807","nonce":2263671727}]},"public_addr":"172.21.15.159:6802/2263671727","cluster_addr":"172.21.15.159:6803/2263671727","heartbeat_back_addr":"172.21.15.159:6806/2263671727","heartbeat_front_addr":"172.21.15.159:6807/2263671727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-27T12:37:55.619 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-06-27T12:37:55.619 INFO:tasks.ceph:Creating RBD pool 2023-06-27T12:37:55.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-27T12:37:55.746 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:55.736+0000 7f6426ddd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:55.749 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:55.746+0000 7f6426ddd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:56.815 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'rbd' created 2023-06-27T12:37:56.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-27T12:37:56.974 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:56.962+0000 7f1c349d6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:56.977 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:56.973+0000 7f1c349d6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:57.822 INFO:teuthology.orchestra.run.smithi138.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-27T12:37:57.837 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-27T12:37:57.838 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-06-27T12:37:57.838 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-06-27T12:37:57.838 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-06-27T12:37:57.839 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-06-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:57.992+0000 7f3f94b91700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:58.002 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:57.998+0000 7f3f94b91700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:58.270 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":4,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2602770118},{"type":"v1","addr":"172.21.15.159:6801","nonce":2602770118}]},"active_addr":"172.21.15.159:6801/2602770118","active_change":"2023-06-27T12:37:51.898143+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.159:0","nonce":4128736144}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.159:0","nonce":4084393167}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.159:0","nonce":4267017935}]}]} 2023-06-27T12:37:58.273 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-06-27T12:37:58.273 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-06-27T12:37:58.274 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-27T12:37:58.404 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:58.397+0000 7f2290e5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:58.407 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:58.404+0000 7f2290e5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:58.614 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:58.615 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":14,"fsid":"0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9","created":"2023-06-27T12:37:45.394408+0000","modified":"2023-06-27T12:37:57.808549+0000","last_up_change":"2023-06-27T12:37:53.788853+0000","last_in_change":"2023-06-27T12:37:47.674294+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-27T12:37:53.945979+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-27T12:37:55.989154+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25bbe3c9-50fb-430c-8ae5-afc94081c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2316736315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":2316736315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6805","nonce":2316736315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2316736315}]},"public_addr":"172.21.15.138:6800/2316736315","cluster_addr":"172.21.15.138:6801/2316736315","heartbeat_back_addr":"172.21.15.138:6805/2316736315","heartbeat_front_addr":"172.21.15.138:6804/2316736315","state":["exists","up"]},{"osd":1,"uuid":"3b06ea8c-fece-4d4b-bafe-f670227862c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":2294192134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2294192134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2294192134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":2294192134}]},"public_addr":"172.21.15.138:6803/2294192134","cluster_addr":"172.21.15.138:6802/2294192134","heartbeat_back_addr":"172.21.15.138:6806/2294192134","heartbeat_front_addr":"172.21.15.138:6807/2294192134","state":["exists","up"]},{"osd":2,"uuid":"ac444bcf-4783-4337-9729-05a1b08d60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1657039289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6805","nonce":1657039289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6809","nonce":1657039289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1657039289}]},"public_addr":"172.21.15.159:6804/1657039289","cluster_addr":"172.21.15.159:6805/1657039289","heartbeat_back_addr":"172.21.15.159:6809/1657039289","heartbeat_front_addr":"172.21.15.159:6808/1657039289","state":["exists","up"]},{"osd":3,"uuid":"f3f5961d-51f1-4b83-9cb4-ce14355d9dd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2263671727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6803","nonce":2263671727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2263671727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6807","nonce":2263671727}]},"public_addr":"172.21.15.159:6802/2263671727","cluster_addr":"172.21.15.159:6803/2263671727","heartbeat_back_addr":"172.21.15.159:6806/2263671727","heartbeat_front_addr":"172.21.15.159:6807/2263671727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-27T12:37:58.629 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-06-27T12:37:58.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-27T12:37:58.761 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:58.753+0000 7f18e7fae700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:58.764 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:58.761+0000 7f18e7fae700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:58.965 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:58.965 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":15,"fsid":"0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9","created":"2023-06-27T12:37:45.394408+0000","modified":"2023-06-27T12:37:58.815122+0000","last_up_change":"2023-06-27T12:37:53.788853+0000","last_in_change":"2023-06-27T12:37:47.674294+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-27T12:37:53.945979+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-27T12:37:55.989154+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25bbe3c9-50fb-430c-8ae5-afc94081c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2316736315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":2316736315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6805","nonce":2316736315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2316736315}]},"public_addr":"172.21.15.138:6800/2316736315","cluster_addr":"172.21.15.138:6801/2316736315","heartbeat_back_addr":"172.21.15.138:6805/2316736315","heartbeat_front_addr":"172.21.15.138:6804/2316736315","state":["exists","up"]},{"osd":1,"uuid":"3b06ea8c-fece-4d4b-bafe-f670227862c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":2294192134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2294192134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2294192134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":2294192134}]},"public_addr":"172.21.15.138:6803/2294192134","cluster_addr":"172.21.15.138:6802/2294192134","heartbeat_back_addr":"172.21.15.138:6806/2294192134","heartbeat_front_addr":"172.21.15.138:6807/2294192134","state":["exists","up"]},{"osd":2,"uuid":"ac444bcf-4783-4337-9729-05a1b08d60bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1657039289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6805","nonce":1657039289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6809","nonce":1657039289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1657039289}]},"public_addr":"172.21.15.159:6804/1657039289","cluster_addr":"172.21.15.159:6805/1657039289","heartbeat_back_addr":"172.21.15.159:6809/1657039289","heartbeat_front_addr":"172.21.15.159:6808/1657039289","state":["exists","up"]},{"osd":3,"uuid":"f3f5961d-51f1-4b83-9cb4-ce14355d9dd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2263671727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6803","nonce":2263671727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2263671727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6807","nonce":2263671727}]},"public_addr":"172.21.15.159:6802/2263671727","cluster_addr":"172.21.15.159:6803/2263671727","heartbeat_back_addr":"172.21.15.159:6806/2263671727","heartbeat_front_addr":"172.21.15.159:6807/2263671727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-27T12:37:58.979 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-06-27T12:37:58.980 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-06-27T12:37:58.981 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-06-27T12:37:58.981 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-06-27T12:37:59.126 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.117+0000 7fa70b3a0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.126+0000 7fa70b3a0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.130 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.121+0000 7f191cfed700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.135 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.132+0000 7f191cfed700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.144 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.135+0000 7fee096b5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.149 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.145+0000 7fee096b5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.152 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.142+0000 7f343b5f3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.158 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.154+0000 7f343b5f3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.167 INFO:teuthology.orchestra.run.smithi138.stdout:42949672965 2023-06-27T12:37:59.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-27T12:37:59.182 INFO:teuthology.orchestra.run.smithi138.stdout:38654705670 2023-06-27T12:37:59.182 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-27T12:37:59.190 INFO:teuthology.orchestra.run.smithi138.stdout:42949672965 2023-06-27T12:37:59.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-27T12:37:59.192 INFO:teuthology.orchestra.run.smithi138.stdout:42949672965 2023-06-27T12:37:59.192 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-27T12:37:59.312 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.303+0000 7f1039705700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.315 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.311+0000 7f1039705700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.342 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.331+0000 7fb50b119700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.346 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.342+0000 7fb50b119700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.379 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.371+0000 7eff1dca0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.382 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.379+0000 7eff1dca0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.394 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.386+0000 7f0e67eec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.398 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.394+0000 7f0e67eec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.529 INFO:teuthology.orchestra.run.smithi138.stdout:42949672965 2023-06-27T12:37:59.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672965 for osd.1 2023-06-27T12:37:59.544 DEBUG:teuthology.parallel:result is None 2023-06-27T12:37:59.582 INFO:teuthology.orchestra.run.smithi138.stdout:38654705670 2023-06-27T12:37:59.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.0 2023-06-27T12:37:59.597 INFO:teuthology.orchestra.run.smithi138.stdout:42949672965 2023-06-27T12:37:59.598 DEBUG:teuthology.parallel:result is None 2023-06-27T12:37:59.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672965 for osd.3 2023-06-27T12:37:59.612 DEBUG:teuthology.parallel:result is None 2023-06-27T12:37:59.622 INFO:teuthology.orchestra.run.smithi138.stdout:42949672965 2023-06-27T12:37:59.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672965 for osd.2 2023-06-27T12:37:59.637 DEBUG:teuthology.parallel:result is None 2023-06-27T12:37:59.637 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-27T12:37:59.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:37:59.803 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.796+0000 7f3398925700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.806 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:37:59.803+0000 7f3398925700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:37:59.997 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:37:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:00.011 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-06-27T12:37:59.905177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107736,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379624,"statfs":{"total":386547056640,"available":386436734976,"internally_reserved":0,"allocated":417792,"data_stored":1854788,"data_compressed":82433,"data_compressed_allocated":110592,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":9,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.906776"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.916103+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:37:57.916103+0000","last_peered":"2023-06-27T12:37:57.916103+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:37:57.916103+0000","last_undegraded":"2023-06-27T12:37:57.916103+0000","last_fullsized":"2023-06-27T12:37:57.916103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.830069+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:37:58.830069+0000","last_peered":"2023-06-27T12:37:58.830069+0000","last_clean":"2023-06-27T12:37:58.830069+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:37:58.830069+0000","last_undegraded":"2023-06-27T12:37:58.830069+0000","last_fullsized":"2023-06-27T12:37:58.830069+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.934379+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:37:57.934379+0000","last_peered":"2023-06-27T12:37:57.934379+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:37:57.934379+0000","last_undegraded":"2023-06-27T12:37:57.934379+0000","last_fullsized":"2023-06-27T12:37:57.934379+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.952219+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:37:57.952219+0000","last_peered":"2023-06-27T12:37:57.952219+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:37:57.952219+0000","last_undegraded":"2023-06-27T12:37:57.952219+0000","last_fullsized":"2023-06-27T12:37:57.952219+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"14'92","reported_seq":136,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:59.140819+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:37:59.140819+0000","last_peered":"2023-06-27T12:37:59.140819+0000","last_clean":"2023-06-27T12:37:59.140819+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:37:59.140819+0000","last_undegraded":"2023-06-27T12:37:59.140819+0000","last_fullsized":"2023-06-27T12:37:59.140819+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.911079+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:37:57.911079+0000","last_peered":"2023-06-27T12:37:57.911079+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:37:57.911079+0000","last_undegraded":"2023-06-27T12:37:57.911079+0000","last_fullsized":"2023-06-27T12:37:57.911079+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.881722+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:37:57.881722+0000","last_peered":"2023-06-27T12:37:57.881722+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:37:57.881722+0000","last_undegraded":"2023-06-27T12:37:57.881722+0000","last_fullsized":"2023-06-27T12:37:57.881722+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.941686+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:37:57.941686+0000","last_peered":"2023-06-27T12:37:57.941686+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:37:57.941686+0000","last_undegraded":"2023-06-27T12:37:57.941686+0000","last_fullsized":"2023-06-27T12:37:57.941686+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.825454+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:37:58.825454+0000","last_peered":"2023-06-27T12:37:58.825454+0000","last_clean":"2023-06-27T12:37:58.825454+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:37:58.825454+0000","last_undegraded":"2023-06-27T12:37:58.825454+0000","last_fullsized":"2023-06-27T12:37:58.825454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26948,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344892,"statfs":{"total":96636764160,"available":96609169408,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27477,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26884,"kb_used_data":60,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344956,"statfs":{"total":96636764160,"available":96609234944,"internally_reserved":0,"allocated":61440,"data_stored":20921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:00.012 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:00.139 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:00.132+0000 7f6c7a4e0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:00.141 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:00.138+0000 7f6c7a4e0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:00.332 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:00.334 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:00.345 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-06-27T12:37:59.905177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107736,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379624,"statfs":{"total":386547056640,"available":386436734976,"internally_reserved":0,"allocated":417792,"data_stored":1854788,"data_compressed":82433,"data_compressed_allocated":110592,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":9,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.906776"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.916103+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:37:57.916103+0000","last_peered":"2023-06-27T12:37:57.916103+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:37:57.916103+0000","last_undegraded":"2023-06-27T12:37:57.916103+0000","last_fullsized":"2023-06-27T12:37:57.916103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.830069+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:37:58.830069+0000","last_peered":"2023-06-27T12:37:58.830069+0000","last_clean":"2023-06-27T12:37:58.830069+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:37:58.830069+0000","last_undegraded":"2023-06-27T12:37:58.830069+0000","last_fullsized":"2023-06-27T12:37:58.830069+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.934379+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:37:57.934379+0000","last_peered":"2023-06-27T12:37:57.934379+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:37:57.934379+0000","last_undegraded":"2023-06-27T12:37:57.934379+0000","last_fullsized":"2023-06-27T12:37:57.934379+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.952219+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:37:57.952219+0000","last_peered":"2023-06-27T12:37:57.952219+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:37:57.952219+0000","last_undegraded":"2023-06-27T12:37:57.952219+0000","last_fullsized":"2023-06-27T12:37:57.952219+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"14'92","reported_seq":136,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:59.140819+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:37:59.140819+0000","last_peered":"2023-06-27T12:37:59.140819+0000","last_clean":"2023-06-27T12:37:59.140819+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:37:59.140819+0000","last_undegraded":"2023-06-27T12:37:59.140819+0000","last_fullsized":"2023-06-27T12:37:59.140819+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.911079+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:37:57.911079+0000","last_peered":"2023-06-27T12:37:57.911079+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:37:57.911079+0000","last_undegraded":"2023-06-27T12:37:57.911079+0000","last_fullsized":"2023-06-27T12:37:57.911079+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.881722+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:37:57.881722+0000","last_peered":"2023-06-27T12:37:57.881722+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:37:57.881722+0000","last_undegraded":"2023-06-27T12:37:57.881722+0000","last_fullsized":"2023-06-27T12:37:57.881722+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.941686+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:37:57.941686+0000","last_peered":"2023-06-27T12:37:57.941686+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:37:57.941686+0000","last_undegraded":"2023-06-27T12:37:57.941686+0000","last_fullsized":"2023-06-27T12:37:57.941686+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.825454+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:37:58.825454+0000","last_peered":"2023-06-27T12:37:58.825454+0000","last_clean":"2023-06-27T12:37:58.825454+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:37:58.825454+0000","last_undegraded":"2023-06-27T12:37:58.825454+0000","last_fullsized":"2023-06-27T12:37:58.825454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26948,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344892,"statfs":{"total":96636764160,"available":96609169408,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27477,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26884,"kb_used_data":60,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344956,"statfs":{"total":96636764160,"available":96609234944,"internally_reserved":0,"allocated":61440,"data_stored":20921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:00.346 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-27T12:38:00.347 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-06-27T12:38:00.347 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-06-27T12:38:00.347 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-06-27T12:38:00.474 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:00.468+0000 7f55e12fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:00.477 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:00.473+0000 7f55e12fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:00.711 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:00.712 INFO:teuthology.orchestra.run.smithi138.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-06-27T12:38:00.725 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-06-27T12:38:00.726 INFO:teuthology.run_tasks:Running task thrashosds... 2023-06-27T12:38:00.735 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2023-06-27T12:38:00.736 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-matanb-crimson-61651'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson seastar smp': 3, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097', 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': 'ba3fff7216d65a7042e2ef64fe924764dc57a097'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'fd62a8dfcb345fabed3eea6b3a461f1ca848a239'}} 2023-06-27T12:38:00.736 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 4, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2023-06-27T12:38:00.736 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-06-27T12:38:00.872 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:00.862+0000 7f0dc450a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:00.875 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:00.872+0000 7f0dc450a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:01.085 INFO:tasks.thrashosds:Beginning thrashosds... 2023-06-27T12:38:01.086 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-27T12:38:01.086 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:01.213 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:01.206+0000 7f194057f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:01.216 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:01.212+0000 7f194057f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:01.406 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:01.409 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:01.420 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-06-27T12:37:59.905177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107736,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379624,"statfs":{"total":386547056640,"available":386436734976,"internally_reserved":0,"allocated":417792,"data_stored":1854788,"data_compressed":82433,"data_compressed_allocated":110592,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":9,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.906776"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.916103+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:37:57.916103+0000","last_peered":"2023-06-27T12:37:57.916103+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:37:57.916103+0000","last_undegraded":"2023-06-27T12:37:57.916103+0000","last_fullsized":"2023-06-27T12:37:57.916103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.830069+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:37:58.830069+0000","last_peered":"2023-06-27T12:37:58.830069+0000","last_clean":"2023-06-27T12:37:58.830069+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:37:58.830069+0000","last_undegraded":"2023-06-27T12:37:58.830069+0000","last_fullsized":"2023-06-27T12:37:58.830069+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.934379+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:37:57.934379+0000","last_peered":"2023-06-27T12:37:57.934379+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:37:57.934379+0000","last_undegraded":"2023-06-27T12:37:57.934379+0000","last_fullsized":"2023-06-27T12:37:57.934379+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.952219+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:37:57.952219+0000","last_peered":"2023-06-27T12:37:57.952219+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:37:57.952219+0000","last_undegraded":"2023-06-27T12:37:57.952219+0000","last_fullsized":"2023-06-27T12:37:57.952219+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"14'92","reported_seq":136,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:59.140819+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:37:59.140819+0000","last_peered":"2023-06-27T12:37:59.140819+0000","last_clean":"2023-06-27T12:37:59.140819+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:37:59.140819+0000","last_undegraded":"2023-06-27T12:37:59.140819+0000","last_fullsized":"2023-06-27T12:37:59.140819+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.911079+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:37:57.911079+0000","last_peered":"2023-06-27T12:37:57.911079+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:37:57.911079+0000","last_undegraded":"2023-06-27T12:37:57.911079+0000","last_fullsized":"2023-06-27T12:37:57.911079+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.881722+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:37:57.881722+0000","last_peered":"2023-06-27T12:37:57.881722+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:37:57.881722+0000","last_undegraded":"2023-06-27T12:37:57.881722+0000","last_fullsized":"2023-06-27T12:37:57.881722+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.941686+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:37:57.941686+0000","last_peered":"2023-06-27T12:37:57.941686+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:37:57.941686+0000","last_undegraded":"2023-06-27T12:37:57.941686+0000","last_fullsized":"2023-06-27T12:37:57.941686+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.825454+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:37:58.825454+0000","last_peered":"2023-06-27T12:37:58.825454+0000","last_clean":"2023-06-27T12:37:58.825454+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:37:58.825454+0000","last_undegraded":"2023-06-27T12:37:58.825454+0000","last_fullsized":"2023-06-27T12:37:58.825454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26948,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344892,"statfs":{"total":96636764160,"available":96609169408,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27477,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26884,"kb_used_data":60,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344956,"statfs":{"total":96636764160,"available":96609234944,"internally_reserved":0,"allocated":61440,"data_stored":20921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:01.422 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:01.549 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:01.542+0000 7f1449911700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:01.551 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:01.548+0000 7f1449911700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:01.741 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:01.743 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:01.755 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-06-27T12:37:59.905177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107736,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379624,"statfs":{"total":386547056640,"available":386436734976,"internally_reserved":0,"allocated":417792,"data_stored":1854788,"data_compressed":82433,"data_compressed_allocated":110592,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":9,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.906776"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.916103+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:37:57.916103+0000","last_peered":"2023-06-27T12:37:57.916103+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:37:57.916103+0000","last_undegraded":"2023-06-27T12:37:57.916103+0000","last_fullsized":"2023-06-27T12:37:57.916103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.830069+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:37:58.830069+0000","last_peered":"2023-06-27T12:37:58.830069+0000","last_clean":"2023-06-27T12:37:58.830069+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:37:58.830069+0000","last_undegraded":"2023-06-27T12:37:58.830069+0000","last_fullsized":"2023-06-27T12:37:58.830069+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.934379+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:37:57.934379+0000","last_peered":"2023-06-27T12:37:57.934379+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:37:57.934379+0000","last_undegraded":"2023-06-27T12:37:57.934379+0000","last_fullsized":"2023-06-27T12:37:57.934379+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.952219+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:37:57.952219+0000","last_peered":"2023-06-27T12:37:57.952219+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:37:57.952219+0000","last_undegraded":"2023-06-27T12:37:57.952219+0000","last_fullsized":"2023-06-27T12:37:57.952219+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"14'92","reported_seq":136,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:59.140819+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:37:59.140819+0000","last_peered":"2023-06-27T12:37:59.140819+0000","last_clean":"2023-06-27T12:37:59.140819+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:37:59.140819+0000","last_undegraded":"2023-06-27T12:37:59.140819+0000","last_fullsized":"2023-06-27T12:37:59.140819+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.911079+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:37:57.911079+0000","last_peered":"2023-06-27T12:37:57.911079+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:37:57.911079+0000","last_undegraded":"2023-06-27T12:37:57.911079+0000","last_fullsized":"2023-06-27T12:37:57.911079+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.881722+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:37:57.881722+0000","last_peered":"2023-06-27T12:37:57.881722+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:37:57.881722+0000","last_undegraded":"2023-06-27T12:37:57.881722+0000","last_fullsized":"2023-06-27T12:37:57.881722+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.941686+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:37:57.941686+0000","last_peered":"2023-06-27T12:37:57.941686+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:37:57.941686+0000","last_undegraded":"2023-06-27T12:37:57.941686+0000","last_fullsized":"2023-06-27T12:37:57.941686+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.825454+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:37:58.825454+0000","last_peered":"2023-06-27T12:37:58.825454+0000","last_clean":"2023-06-27T12:37:58.825454+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:37:58.825454+0000","last_undegraded":"2023-06-27T12:37:58.825454+0000","last_fullsized":"2023-06-27T12:37:58.825454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26948,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344892,"statfs":{"total":96636764160,"available":96609169408,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27477,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26884,"kb_used_data":60,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344956,"statfs":{"total":96636764160,"available":96609234944,"internally_reserved":0,"allocated":61440,"data_stored":20921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:01.756 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-27T12:38:01.756 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-06-27T12:38:01.886 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:01.879+0000 7f2afb8ad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:01.889 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:01.885+0000 7f2afb8ad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:02.085 INFO:teuthology.orchestra.run.smithi138.stdout:epoch 15 2023-06-27T12:38:02.086 INFO:teuthology.orchestra.run.smithi138.stdout:fsid 0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9 2023-06-27T12:38:02.086 INFO:teuthology.orchestra.run.smithi138.stdout:created 2023-06-27T12:37:45.394408+0000 2023-06-27T12:38:02.086 INFO:teuthology.orchestra.run.smithi138.stdout:modified 2023-06-27T12:37:58.815122+0000 2023-06-27T12:38:02.086 INFO:teuthology.orchestra.run.smithi138.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-06-27T12:38:02.086 INFO:teuthology.orchestra.run.smithi138.stdout:crush_version 4 2023-06-27T12:38:02.087 INFO:teuthology.orchestra.run.smithi138.stdout:full_ratio 0.95 2023-06-27T12:38:02.087 INFO:teuthology.orchestra.run.smithi138.stdout:backfillfull_ratio 0.9 2023-06-27T12:38:02.087 INFO:teuthology.orchestra.run.smithi138.stdout:nearfull_ratio 0.85 2023-06-27T12:38:02.087 INFO:teuthology.orchestra.run.smithi138.stdout:require_min_compat_client luminous 2023-06-27T12:38:02.087 INFO:teuthology.orchestra.run.smithi138.stdout:min_compat_client jewel 2023-06-27T12:38:02.087 INFO:teuthology.orchestra.run.smithi138.stdout:require_osd_release reef 2023-06-27T12:38:02.088 INFO:teuthology.orchestra.run.smithi138.stdout:stretch_mode_enabled false 2023-06-27T12:38:02.088 INFO:teuthology.orchestra.run.smithi138.stdout:allow_crimson=true 2023-06-27T12:38:02.088 INFO:teuthology.orchestra.run.smithi138.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 13 flags hashpspool,nopgchange,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 4.00 2023-06-27T12:38:02.088 INFO:teuthology.orchestra.run.smithi138.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 15 flags hashpspool,nopgchange,crimson stripe_width 0 application rbd read_balance_score 1.50 2023-06-27T12:38:02.088 INFO:teuthology.orchestra.run.smithi138.stdout:max_osd 4 2023-06-27T12:38:02.088 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.138:6800/2316736315 v2:172.21.15.138:6801/2316736315 exists,up 25bbe3c9-50fb-430c-8ae5-afc94081c458 2023-06-27T12:38:02.089 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 up in weight 1 up_from 10 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.138:6803/2294192134 v2:172.21.15.138:6802/2294192134 exists,up 3b06ea8c-fece-4d4b-bafe-f670227862c9 2023-06-27T12:38:02.089 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2 up in weight 1 up_from 10 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.159:6804/1657039289 v2:172.21.15.159:6805/1657039289 exists,up ac444bcf-4783-4337-9729-05a1b08d60bc 2023-06-27T12:38:02.089 INFO:teuthology.orchestra.run.smithi138.stdout:osd.3 up in weight 1 up_from 10 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.159:6802/2263671727 v2:172.21.15.159:6803/2263671727 exists,up f3f5961d-51f1-4b83-9cb4-ce14355d9dd6 2023-06-27T12:38:02.099 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.138:6800/2316736315 v2:172.21.15.138:6801/2316736315 exists,up 25bbe3c9-50fb-430c-8ae5-afc94081c458', 'osd.1 up in weight 1 up_from 10 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.138:6803/2294192134 v2:172.21.15.138:6802/2294192134 exists,up 3b06ea8c-fece-4d4b-bafe-f670227862c9', 'osd.2 up in weight 1 up_from 10 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.159:6804/1657039289 v2:172.21.15.159:6805/1657039289 exists,up ac444bcf-4783-4337-9729-05a1b08d60bc', 'osd.3 up in weight 1 up_from 10 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.159:6802/2263671727 v2:172.21.15.159:6803/2263671727 exists,up f3f5961d-51f1-4b83-9cb4-ce14355d9dd6'] 2023-06-27T12:38:02.100 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-06-27T12:38:02.226 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2023-06-27T12:38:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mon.a", 2023-06-27T12:38:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster": "ceph", 2023-06-27T12:38:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-06-27T12:38:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "admin_socket_mode": "", 2023-06-27T12:38:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-06-27T12:38:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_client_required": "cephx, none", 2023-06-27T12:38:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_cluster_required": "cephx", 2023-06-27T12:38:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_debug": "true", 2023-06-27T12:38:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-06-27T12:38:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-06-27T12:38:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_service_required": "cephx", 2023-06-27T12:38:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_service_ticket_ttl": "240.000000", 2023-06-27T12:38:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_supported": "cephx", 2023-06-27T12:38:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_aio": "true", 2023-06-27T12:38:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_aio_max_queue_depth": "1024", 2023-06-27T12:38:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_aio_poll_ms": "250", 2023-06-27T12:38:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_aio_reap_max": "16", 2023-06-27T12:38:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_async_discard": "false", 2023-06-27T12:38:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_block_size": "4096", 2023-06-27T12:38:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_debug_aio": "false", 2023-06-27T12:38:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-06-27T12:38:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-06-27T12:38:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_debug_inflight_ios": "false", 2023-06-27T12:38:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_enable_discard": "false", 2023-06-27T12:38:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_flock_retry": "3", 2023-06-27T12:38:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_flock_retry_interval": "0.100000", 2023-06-27T12:38:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_inject_crash": "0", 2023-06-27T12:38:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_inject_crash_flush_delay": "2", 2023-06-27T12:38:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_ioring": "false", 2023-06-27T12:38:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_ioring_hipri": "false", 2023-06-27T12:38:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_ioring_sqthread_poll": "false", 2023-06-27T12:38:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-06-27T12:38:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_read_buffer_alignment": "4096", 2023-06-27T12:38:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-06-27T12:38:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_type": "", 2023-06-27T12:38:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_alloc_size": "1048576", 2023-06-27T12:38:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_allocator": "hybrid", 2023-06-27T12:38:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_buffered_io": "true", 2023-06-27T12:38:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_check_for_zeros": "false", 2023-06-27T12:38:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_check_volume_selector_often": "false", 2023-06-27T12:38:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-06-27T12:38:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_compact_log_sync": "false", 2023-06-27T12:38:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2023-06-27T12:38:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-06-27T12:38:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_log_compact_min_size": "16777216", 2023-06-27T12:38:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_log_replay_check_allocations": "true", 2023-06-27T12:38:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_max_log_runway": "4194304", 2023-06-27T12:38:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_max_prefetch": "1048576", 2023-06-27T12:38:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_min_flush_size": "524288", 2023-06-27T12:38:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_min_log_runway": "1048576", 2023-06-27T12:38:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_replay_recovery": "false", 2023-06-27T12:38:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-06-27T12:38:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_shared_alloc_size": "65536", 2023-06-27T12:38:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_sync_write": "false", 2023-06-27T12:38:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-06-27T12:38:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-06-27T12:38:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-06-27T12:38:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_allocation_from_file": "true", 2023-06-27T12:38:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_allocator": "hybrid", 2023-06-27T12:38:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-06-27T12:38:02.235 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-06-27T12:38:02.235 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-06-27T12:38:02.235 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-06-27T12:38:02.235 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-06-27T12:38:02.235 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-06-27T12:38:02.235 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blobid_prealloc": "10240", 2023-06-27T12:38:02.235 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_create": "true", 2023-06-27T12:38:02.235 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_db_create": "false", 2023-06-27T12:38:02.236 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_db_path": "", 2023-06-27T12:38:02.236 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_db_size": "0", 2023-06-27T12:38:02.236 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_path": "", 2023-06-27T12:38:02.236 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_preallocate_file": "false", 2023-06-27T12:38:02.236 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_size": "107374182400", 2023-06-27T12:38:02.236 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_wal_create": "false", 2023-06-27T12:38:02.236 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_wal_path": "", 2023-06-27T12:38:02.237 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_wal_size": "100663296", 2023-06-27T12:38:02.237 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bluefs": "true", 2023-06-27T12:38:02.237 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-06-27T12:38:02.237 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bluefs_env_mirror": "false", 2023-06-27T12:38:02.237 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-06-27T12:38:02.237 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-06-27T12:38:02.237 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-06-27T12:38:02.237 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-06-27T12:38:02.238 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-06-27T12:38:02.238 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-06-27T12:38:02.238 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_autotune": "true", 2023-06-27T12:38:02.238 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-06-27T12:38:02.238 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-06-27T12:38:02.238 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-06-27T12:38:02.238 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-06-27T12:38:02.238 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_size": "0", 2023-06-27T12:38:02.239 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-06-27T12:38:02.239 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-06-27T12:38:02.239 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-06-27T12:38:02.239 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-06-27T12:38:02.239 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_type": "2q", 2023-06-27T12:38:02.239 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-06-27T12:38:02.239 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_clone_cow": "true", 2023-06-27T12:38:02.239 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_algorithm": "snappy", 2023-06-27T12:38:02.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_max_blob_size": "0", 2023-06-27T12:38:02.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-06-27T12:38:02.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-06-27T12:38:02.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_min_blob_size": "0", 2023-06-27T12:38:02.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-06-27T12:38:02.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-06-27T12:38:02.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_mode": "none", 2023-06-27T12:38:02.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-06-27T12:38:02.241 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_csum_type": "crc32c", 2023-06-27T12:38:02.241 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_enforce_settings": "default", 2023-06-27T12:38:02.241 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_freelist": "false", 2023-06-27T12:38:02.241 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_fsck_abort": "false", 2023-06-27T12:38:02.241 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-06-27T12:38:02.241 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_inject_bug21040": "false", 2023-06-27T12:38:02.241 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-06-27T12:38:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_inject_read_err": "false", 2023-06-27T12:38:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_legacy_omap": "false", 2023-06-27T12:38:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-06-27T12:38:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-06-27T12:38:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-06-27T12:38:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-06-27T12:38:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_prefill": "0.000000", 2023-06-27T12:38:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-06-27T12:38:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-06-27T12:38:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-06-27T12:38:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_small_allocations": "0", 2023-06-27T12:38:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-06-27T12:38:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_default_buffered_read": "true", 2023-06-27T12:38:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_default_buffered_write": "false", 2023-06-27T12:38:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_deferred_batch_ops": "0", 2023-06-27T12:38:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-06-27T12:38:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-06-27T12:38:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-06-27T12:38:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-06-27T12:38:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-06-27T12:38:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-06-27T12:38:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-06-27T12:38:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fail_eio": "false", 2023-06-27T12:38:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-06-27T12:38:02.245 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-06-27T12:38:02.245 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-06-27T12:38:02.245 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-06-27T12:38:02.245 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_mkfs": "true", 2023-06-27T12:38:02.245 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-06-27T12:38:02.245 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_mount": "false", 2023-06-27T12:38:02.245 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-06-27T12:38:02.245 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_umount": "false", 2023-06-27T12:38:02.246 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-06-27T12:38:02.246 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-06-27T12:38:02.246 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-06-27T12:38:02.246 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-06-27T12:38:02.246 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-06-27T12:38:02.246 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-06-27T12:38:02.246 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-06-27T12:38:02.246 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-06-27T12:38:02.247 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_ignore_data_csum": "false", 2023-06-27T12:38:02.247 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-06-27T12:38:02.247 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_kvbackend": "rocksdb", 2023-06-27T12:38:02.247 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-06-27T12:38:02.247 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-06-27T12:38:02.247 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_log_op_age": "5.000000", 2023-06-27T12:38:02.247 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_alloc_size": "0", 2023-06-27T12:38:02.247 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_blob_size": "0", 2023-06-27T12:38:02.248 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-06-27T12:38:02.248 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-06-27T12:38:02.248 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_defer_interval": "3.000000", 2023-06-27T12:38:02.248 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_deferred_txc": "32", 2023-06-27T12:38:02.248 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_min_alloc_size": "0", 2023-06-27T12:38:02.248 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-06-27T12:38:02.248 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-06-27T12:38:02.248 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_nid_prealloc": "1024", 2023-06-27T12:38:02.249 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_prefer_deferred_size": "0", 2023-06-27T12:38:02.249 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-06-27T12:38:02.249 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-06-27T12:38:02.249 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_qfsck_on_mount": "true", 2023-06-27T12:38:02.249 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_retry_disk_reads": "3", 2023-06-27T12:38:02.249 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_rocksdb_cf": "true", 2023-06-27T12:38:02.249 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2023-06-27T12:38:02.250 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2023-06-27T12:38:02.250 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_rocksdb_options_annex": "", 2023-06-27T12:38:02.250 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_spdk_coremask": "0x1", 2023-06-27T12:38:02.250 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_spdk_io_sleep": "5", 2023-06-27T12:38:02.250 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_spdk_max_io_completion": "0", 2023-06-27T12:38:02.250 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_spdk_mem": "512", 2023-06-27T12:38:02.250 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_sync_submit_transaction": "false", 2023-06-27T12:38:02.250 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_bytes": "67108864", 2023-06-27T12:38:02.251 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_cost_per_io": "0", 2023-06-27T12:38:02.251 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-06-27T12:38:02.251 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-06-27T12:38:02.251 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-06-27T12:38:02.251 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-06-27T12:38:02.251 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_tracing": "false", 2023-06-27T12:38:02.251 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-06-27T12:38:02.251 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-06-27T12:38:02.252 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_volume_selection_reserved": "0", 2023-06-27T12:38:02.252 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-06-27T12:38:02.252 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-06-27T12:38:02.252 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-06-27T12:38:02.252 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-06-27T12:38:02.252 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-06-27T12:38:02.252 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-06-27T12:38:02.253 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_zero_block_detection": "false", 2023-06-27T12:38:02.253 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-06-27T12:38:02.253 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_action_update_interval": "2", 2023-06-27T12:38:02.253 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-06-27T12:38:02.253 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-06-27T12:38:02.253 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-06-27T12:38:02.253 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-06-27T12:38:02.253 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_mount_timeout": "10", 2023-06-27T12:38:02.254 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-06-27T12:38:02.254 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-06-27T12:38:02.254 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-06-27T12:38:02.254 INFO:teuthology.orchestra.run.smithi138.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-06-27T12:38:02.254 INFO:teuthology.orchestra.run.smithi138.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-06-27T12:38:02.254 INFO:teuthology.orchestra.run.smithi138.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-06-27T12:38:02.254 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_cluster_require_signatures": "false", 2023-06-27T12:38:02.254 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_cluster_require_version": "2", 2023-06-27T12:38:02.255 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_require_signatures": "false", 2023-06-27T12:38:02.255 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_require_version": "2", 2023-06-27T12:38:02.255 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_service_require_signatures": "false", 2023-06-27T12:38:02.255 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_service_require_version": "2", 2023-06-27T12:38:02.255 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_sign_messages": "true", 2023-06-27T12:38:02.255 INFO:teuthology.orchestra.run.smithi138.stdout: "chdir": "", 2023-06-27T12:38:02.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client_acl_type": "", 2023-06-27T12:38:02.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client_asio_thread_count": "2", 2023-06-27T12:38:02.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_cache_mid": "0.750000", 2023-06-27T12:38:02.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_cache_size": "16384", 2023-06-27T12:38:02.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_caps_release_delay": "5", 2023-06-27T12:38:02.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_check_pool_perm": "true", 2023-06-27T12:38:02.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_collect_and_send_global_metrics": "false", 2023-06-27T12:38:02.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_debug_force_sync_read": "false", 2023-06-27T12:38:02.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_debug_getattr_caps": "false", 2023-06-27T12:38:02.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_debug_inject_tick_delay": "0", 2023-06-27T12:38:02.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-06-27T12:38:02.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_die_on_failed_remount": "false", 2023-06-27T12:38:02.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_dirsize_rbytes": "true", 2023-06-27T12:38:02.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_force_lazyio": "false", 2023-06-27T12:38:02.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_fs": "", 2023-06-27T12:38:02.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_inject_fixed_oldest_tid": "false", 2023-06-27T12:38:02.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_inject_release_failure": "false", 2023-06-27T12:38:02.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_max_inline_size": "4096", 2023-06-27T12:38:02.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_max_retries_on_remount_failure": "5", 2023-06-27T12:38:02.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mds_namespace": "", 2023-06-27T12:38:02.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_metadata": "", 2023-06-27T12:38:02.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mount_gid": "-1", 2023-06-27T12:38:02.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mount_timeout": "300", 2023-06-27T12:38:02.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mount_uid": "-1", 2023-06-27T12:38:02.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mountpoint": "/", 2023-06-27T12:38:02.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_notify_timeout": "10", 2023-06-27T12:38:02.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc": "true", 2023-06-27T12:38:02.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_max_dirty": "104857600", 2023-06-27T12:38:02.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_max_dirty_age": "5.000000", 2023-06-27T12:38:02.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_max_objects": "1000", 2023-06-27T12:38:02.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_size": "209715200", 2023-06-27T12:38:02.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_target_dirty": "8388608", 2023-06-27T12:38:02.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_permissions": "true", 2023-06-27T12:38:02.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_quota": "true", 2023-06-27T12:38:02.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_quota_df": "true", 2023-06-27T12:38:02.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_readahead_max_bytes": "0", 2023-06-27T12:38:02.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_readahead_max_periods": "4", 2023-06-27T12:38:02.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_readahead_min": "131072", 2023-06-27T12:38:02.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_reconnect_stale": "false", 2023-06-27T12:38:02.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_shutdown_timeout": "30", 2023-06-27T12:38:02.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_snapdir": ".snap", 2023-06-27T12:38:02.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_tick_interval": "1", 2023-06-27T12:38:02.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_trace": "", 2023-06-27T12:38:02.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_try_dentry_invalidate": "false", 2023-06-27T12:38:02.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_use_faked_inos": "false", 2023-06-27T12:38:02.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_use_random_mds": "false", 2023-06-27T12:38:02.261 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_graylog": "false", 2023-06-27T12:38:02.261 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-06-27T12:38:02.261 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_graylog_port": "12201", 2023-06-27T12:38:02.262 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_monitors": "default=true", 2023-06-27T12:38:02.262 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_syslog": "false", 2023-06-27T12:38:02.262 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-06-27T12:38:02.262 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_syslog_level": "info", 2023-06-27T12:38:02.262 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "-", 2023-06-27T12:38:02.262 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_network": "", 2023-06-27T12:38:02.262 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_network_interface": "", 2023-06-27T12:38:02.262 INFO:teuthology.orchestra.run.smithi138.stdout: "compressor_zlib_isal": "false", 2023-06-27T12:38:02.263 INFO:teuthology.orchestra.run.smithi138.stdout: "compressor_zlib_level": "5", 2023-06-27T12:38:02.263 INFO:teuthology.orchestra.run.smithi138.stdout: "compressor_zlib_winsize": "-15", 2023-06-27T12:38:02.263 INFO:teuthology.orchestra.run.smithi138.stdout: "compressor_zstd_level": "1", 2023-06-27T12:38:02.263 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-06-27T12:38:02.263 INFO:teuthology.orchestra.run.smithi138.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-06-27T12:38:02.264 INFO:teuthology.orchestra.run.smithi138.stdout: "crimson_alien_op_num_threads": "6", 2023-06-27T12:38:02.264 INFO:teuthology.orchestra.run.smithi138.stdout: "crimson_alien_thread_cpu_cores": "", 2023-06-27T12:38:02.264 INFO:teuthology.orchestra.run.smithi138.stdout: "crimson_osd_obc_lru_size": "10", 2023-06-27T12:38:02.264 INFO:teuthology.orchestra.run.smithi138.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-06-27T12:38:02.264 INFO:teuthology.orchestra.run.smithi138.stdout: "crimson_seastar_smp": "1", 2023-06-27T12:38:02.264 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_location": "", 2023-06-27T12:38:02.264 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_location_hook": "", 2023-06-27T12:38:02.264 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_location_hook_timeout": "10", 2023-06-27T12:38:02.265 INFO:teuthology.orchestra.run.smithi138.stdout: "daemonize": "false", 2023-06-27T12:38:02.265 INFO:teuthology.orchestra.run.smithi138.stdout: "daos_pool": "tank", 2023-06-27T12:38:02.265 INFO:teuthology.orchestra.run.smithi138.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-06-27T12:38:02.265 INFO:teuthology.orchestra.run.smithi138.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-06-27T12:38:02.265 INFO:teuthology.orchestra.run.smithi138.stdout: "dbstore_db_name_prefix": "dbstore", 2023-06-27T12:38:02.265 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_alienstore": "0/5", 2023-06-27T12:38:02.265 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_allow_any_pool_priority": "false", 2023-06-27T12:38:02.265 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_asok": "1/5", 2023-06-27T12:38:02.266 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_asok_assert_abort": "false", 2023-06-27T12:38:02.266 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_asserts_on_shutdown": "true", 2023-06-27T12:38:02.266 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_auth": "20/20", 2023-06-27T12:38:02.266 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_bdev": "1/3", 2023-06-27T12:38:02.266 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_bluefs": "1/5", 2023-06-27T12:38:02.266 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_bluestore": "1/5", 2023-06-27T12:38:02.266 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_buffer": "0/1", 2023-06-27T12:38:02.266 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_ceph_exporter": "1/5", 2023-06-27T12:38:02.267 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_cephfs_mirror": "0/5", 2023-06-27T12:38:02.267 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_cephsqlite": "0/5", 2023-06-27T12:38:02.267 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_client": "0/5", 2023-06-27T12:38:02.267 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_compressor": "1/5", 2023-06-27T12:38:02.267 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_context": "0/1", 2023-06-27T12:38:02.267 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_crush": "1/1", 2023-06-27T12:38:02.267 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_crypto": "1/5", 2023-06-27T12:38:02.267 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_cyanstore": "0/5", 2023-06-27T12:38:02.268 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_deliberately_leak_memory": "false", 2023-06-27T12:38:02.268 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_disable_randomized_ping": "false", 2023-06-27T12:38:02.268 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_dpdk": "1/5", 2023-06-27T12:38:02.268 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_eventtrace": "1/5", 2023-06-27T12:38:02.268 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_filer": "0/1", 2023-06-27T12:38:02.268 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_filestore": "1/3", 2023-06-27T12:38:02.268 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_finisher": "1/1", 2023-06-27T12:38:02.268 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_fuse": "1/5", 2023-06-27T12:38:02.269 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_heartbeat_testing_span": "0", 2023-06-27T12:38:02.269 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_heartbeatmap": "1/5", 2023-06-27T12:38:02.269 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_immutable_obj_cache": "0/5", 2023-06-27T12:38:02.269 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_javaclient": "1/5", 2023-06-27T12:38:02.269 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_journal": "1/3", 2023-06-27T12:38:02.269 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_journaler": "0/5", 2023-06-27T12:38:02.269 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_kstore": "1/5", 2023-06-27T12:38:02.269 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_lockdep": "0/1", 2023-06-27T12:38:02.270 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mclock": "1/5", 2023-06-27T12:38:02.270 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds": "1/5", 2023-06-27T12:38:02.270 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_balancer": "1/5", 2023-06-27T12:38:02.270 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_locker": "1/5", 2023-06-27T12:38:02.270 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_log": "1/5", 2023-06-27T12:38:02.270 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_log_expire": "1/5", 2023-06-27T12:38:02.270 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_migrator": "1/5", 2023-06-27T12:38:02.271 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_memstore": "1/5", 2023-06-27T12:38:02.271 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mgr": "2/5", 2023-06-27T12:38:02.271 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mgrc": "1/5", 2023-06-27T12:38:02.271 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mon": "20/20", 2023-06-27T12:38:02.271 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_monc": "0/10", 2023-06-27T12:38:02.271 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_ms": "1/1", 2023-06-27T12:38:02.271 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_none": "0/5", 2023-06-27T12:38:02.271 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_objclass": "0/5", 2023-06-27T12:38:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_objectcacher": "0/5", 2023-06-27T12:38:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_objecter": "0/1", 2023-06-27T12:38:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_optracker": "0/5", 2023-06-27T12:38:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_osd": "1/5", 2023-06-27T12:38:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_paxos": "20/20", 2023-06-27T12:38:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_perfcounter": "1/5", 2023-06-27T12:38:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_prioritycache": "1/5", 2023-06-27T12:38:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rados": "0/5", 2023-06-27T12:38:02.273 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rbd": "0/5", 2023-06-27T12:38:02.273 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rbd_mirror": "0/5", 2023-06-27T12:38:02.273 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rbd_pwl": "0/5", 2023-06-27T12:38:02.273 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rbd_replay": "0/5", 2023-06-27T12:38:02.273 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_refs": "0/0", 2023-06-27T12:38:02.273 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_reserver": "1/1", 2023-06-27T12:38:02.273 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rgw": "1/5", 2023-06-27T12:38:02.273 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rgw_access": "1/5", 2023-06-27T12:38:02.274 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rgw_datacache": "1/5", 2023-06-27T12:38:02.274 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rgw_dbstore": "1/5", 2023-06-27T12:38:02.274 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rgw_flight": "1/5", 2023-06-27T12:38:02.274 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rgw_sync": "1/5", 2023-06-27T12:38:02.274 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rocksdb": "4/5", 2023-06-27T12:38:02.274 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore": "0/5", 2023-06-27T12:38:02.274 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_backref": "0/5", 2023-06-27T12:38:02.274 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_cache": "0/5", 2023-06-27T12:38:02.275 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_cleaner": "0/5", 2023-06-27T12:38:02.275 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_device": "0/5", 2023-06-27T12:38:02.275 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_epm": "0/5", 2023-06-27T12:38:02.275 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-06-27T12:38:02.275 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_journal": "0/5", 2023-06-27T12:38:02.275 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_lba": "0/5", 2023-06-27T12:38:02.275 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_odata": "0/5", 2023-06-27T12:38:02.275 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_omap": "0/5", 2023-06-27T12:38:02.276 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_onode": "0/5", 2023-06-27T12:38:02.276 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_t": "0/5", 2023-06-27T12:38:02.276 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_seastore_tm": "0/5", 2023-06-27T12:38:02.276 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_striper": "0/1", 2023-06-27T12:38:02.276 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_test": "0/5", 2023-06-27T12:38:02.276 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_throttle": "1/1", 2023-06-27T12:38:02.276 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_timer": "0/1", 2023-06-27T12:38:02.276 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_tp": "0/5", 2023-06-27T12:38:02.277 INFO:teuthology.orchestra.run.smithi138.stdout: "device_failure_prediction_mode": "none", 2023-06-27T12:38:02.277 INFO:teuthology.orchestra.run.smithi138.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "crimson", 2023-06-27T12:38:02.277 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-06-27T12:38:02.277 INFO:teuthology.orchestra.run.smithi138.stdout: "err_to_graylog": "false", 2023-06-27T12:38:02.277 INFO:teuthology.orchestra.run.smithi138.stdout: "err_to_journald": "false", 2023-06-27T12:38:02.277 INFO:teuthology.orchestra.run.smithi138.stdout: "err_to_stderr": "true", 2023-06-27T12:38:02.277 INFO:teuthology.orchestra.run.smithi138.stdout: "err_to_syslog": "false", 2023-06-27T12:38:02.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event_tracing": "false", 2023-06-27T12:38:02.278 INFO:teuthology.orchestra.run.smithi138.stdout: "exporter_addr": "0.0.0.0", 2023-06-27T12:38:02.278 INFO:teuthology.orchestra.run.smithi138.stdout: "exporter_http_port": "9926", 2023-06-27T12:38:02.278 INFO:teuthology.orchestra.run.smithi138.stdout: "exporter_prio_limit": "5", 2023-06-27T12:38:02.278 INFO:teuthology.orchestra.run.smithi138.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-06-27T12:38:02.278 INFO:teuthology.orchestra.run.smithi138.stdout: "exporter_sort_metrics": "true", 2023-06-27T12:38:02.278 INFO:teuthology.orchestra.run.smithi138.stdout: "exporter_stats_period": "5", 2023-06-27T12:38:02.278 INFO:teuthology.orchestra.run.smithi138.stdout: "fake_statfs_for_testing": "0", 2023-06-27T12:38:02.279 INFO:teuthology.orchestra.run.smithi138.stdout: "fatal_signal_handlers": "true", 2023-06-27T12:38:02.279 INFO:teuthology.orchestra.run.smithi138.stdout: "filer_max_purge_ops": "10", 2023-06-27T12:38:02.279 INFO:teuthology.orchestra.run.smithi138.stdout: "filer_max_truncate_ops": "128", 2023-06-27T12:38:02.279 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_apply_finisher_threads": "1", 2023-06-27T12:38:02.279 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_blackhole": "false", 2023-06-27T12:38:02.279 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_btrfs_clone_range": "true", 2023-06-27T12:38:02.279 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_btrfs_snap": "true", 2023-06-27T12:38:02.279 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_caller_concurrency": "10", 2023-06-27T12:38:02.280 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_collect_device_partition_information": "true", 2023-06-27T12:38:02.280 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_commit_timeout": "600.000000", 2023-06-27T12:38:02.280 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_debug_inject_read_err": "false", 2023-06-27T12:38:02.280 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_debug_omap_check": "false", 2023-06-27T12:38:02.280 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_debug_random_read_err": "0.000000", 2023-06-27T12:38:02.280 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_debug_verify_split": "false", 2023-06-27T12:38:02.280 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_dump_file": "", 2023-06-27T12:38:02.280 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-06-27T12:38:02.281 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-06-27T12:38:02.281 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fadvise": "true", 2023-06-27T12:38:02.281 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fail_eio": "true", 2023-06-27T12:38:02.281 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fd_cache_shards": "16", 2023-06-27T12:38:02.281 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fd_cache_size": "128", 2023-06-27T12:38:02.281 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fiemap": "false", 2023-06-27T12:38:02.281 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fiemap_threshold": "4096", 2023-06-27T12:38:02.282 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-06-27T12:38:02.282 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_index_retry_probability": "0.000000", 2023-06-27T12:38:02.282 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_inject_stall": "0", 2023-06-27T12:38:02.282 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_journal_parallel": "false", 2023-06-27T12:38:02.282 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_journal_trailing": "false", 2023-06-27T12:38:02.282 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_journal_writeahead": "false", 2023-06-27T12:38:02.282 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_kill_at": "0", 2023-06-27T12:38:02.282 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-06-27T12:38:02.283 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattr_size": "0", 2023-06-27T12:38:02.283 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-06-27T12:38:02.283 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-06-27T12:38:02.283 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-06-27T12:38:02.283 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattrs": "0", 2023-06-27T12:38:02.283 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-06-27T12:38:02.283 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattrs_other": "2", 2023-06-27T12:38:02.283 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-06-27T12:38:02.284 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_sync_interval": "5.000000", 2023-06-27T12:38:02.284 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_xattr_value_size": "0", 2023-06-27T12:38:02.284 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-06-27T12:38:02.284 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-06-27T12:38:02.284 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-06-27T12:38:02.284 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_merge_threshold": "-10", 2023-06-27T12:38:02.284 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_min_sync_interval": "0.010000", 2023-06-27T12:38:02.284 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_odsync_write": "false", 2023-06-27T12:38:02.285 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_omap_backend": "rocksdb", 2023-06-27T12:38:02.285 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_omap_backend_path": "", 2023-06-27T12:38:02.285 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_omap_header_cache_size": "1024", 2023-06-27T12:38:02.285 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_ondisk_finisher_threads": "1", 2023-06-27T12:38:02.285 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-06-27T12:38:02.285 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_op_thread_timeout": "60", 2023-06-27T12:38:02.285 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_op_threads": "2", 2023-06-27T12:38:02.285 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_punch_hole": "false", 2023-06-27T12:38:02.286 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-06-27T12:38:02.286 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-06-27T12:38:02.286 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-06-27T12:38:02.286 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-06-27T12:38:02.286 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-06-27T12:38:02.286 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_bytes": "104857600", 2023-06-27T12:38:02.286 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-06-27T12:38:02.287 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-06-27T12:38:02.287 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-06-27T12:38:02.287 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_ops": "50", 2023-06-27T12:38:02.287 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-06-27T12:38:02.287 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_seek_data_hole": "false", 2023-06-27T12:38:02.287 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_sloppy_crc": "false", 2023-06-27T12:38:02.287 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-06-27T12:38:02.287 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_splice": "false", 2023-06-27T12:38:02.288 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_split_multiple": "2", 2023-06-27T12:38:02.288 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_split_rand_factor": "20", 2023-06-27T12:38:02.288 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_update_to": "1000", 2023-06-27T12:38:02.288 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-06-27T12:38:02.288 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-06-27T12:38:02.288 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-06-27T12:38:02.288 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-06-27T12:38:02.288 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-06-27T12:38:02.289 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-06-27T12:38:02.289 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_enable": "true", 2023-06-27T12:38:02.289 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-06-27T12:38:02.289 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-06-27T12:38:02.289 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-06-27T12:38:02.289 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-06-27T12:38:02.289 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-06-27T12:38:02.289 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-06-27T12:38:02.290 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_xfs_extsize": "false", 2023-06-27T12:38:02.290 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_zfs_snap": "false", 2023-06-27T12:38:02.290 INFO:teuthology.orchestra.run.smithi138.stdout: "fio_dir": "/tmp/fio", 2023-06-27T12:38:02.290 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "0a74ff7b-ccd7-40da-8eaf-76b3078fe9e9", 2023-06-27T12:38:02.290 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_allow_other": "true", 2023-06-27T12:38:02.290 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_atomic_o_trunc": "true", 2023-06-27T12:38:02.290 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_big_writes": "true", 2023-06-27T12:38:02.290 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_debug": "false", 2023-06-27T12:38:02.291 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_default_permissions": "false", 2023-06-27T12:38:02.291 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_disable_pagecache": "false", 2023-06-27T12:38:02.291 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_max_write": "0", 2023-06-27T12:38:02.291 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_multithreaded": "true", 2023-06-27T12:38:02.291 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_require_active_mds": "true", 2023-06-27T12:38:02.291 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_set_user_groups": "true", 2023-06-27T12:38:02.291 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_splice_move": "true", 2023-06-27T12:38:02.291 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_splice_read": "true", 2023-06-27T12:38:02.292 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_splice_write": "true", 2023-06-27T12:38:02.292 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_syncfs_on_mksnap": "true", 2023-06-27T12:38:02.292 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_use_invalidate_cb": "true", 2023-06-27T12:38:02.292 INFO:teuthology.orchestra.run.smithi138.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-06-27T12:38:02.292 INFO:teuthology.orchestra.run.smithi138.stdout: "gss_target_name": "ceph", 2023-06-27T12:38:02.292 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_file": "", 2023-06-27T12:38:02.292 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_inject_failure": "0", 2023-06-27T12:38:02.293 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_interval": "5", 2023-06-27T12:38:02.293 INFO:teuthology.orchestra.run.smithi138.stdout: "host": "smithi138", 2023-06-27T12:38:02.293 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-06-27T12:38:02.293 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-06-27T12:38:02.293 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_max_size": "1073741824", 2023-06-27T12:38:02.293 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-06-27T12:38:02.293 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-06-27T12:38:02.293 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-06-27T12:38:02.294 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-06-27T12:38:02.294 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-06-27T12:38:02.294 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-06-27T12:38:02.294 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-06-27T12:38:02.294 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-06-27T12:38:02.294 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-06-27T12:38:02.294 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_watermark": "0.900000", 2023-06-27T12:38:02.294 INFO:teuthology.orchestra.run.smithi138.stdout: "inject_early_sigterm": "false", 2023-06-27T12:38:02.295 INFO:teuthology.orchestra.run.smithi138.stdout: "jaeger_agent_port": "6799", 2023-06-27T12:38:02.295 INFO:teuthology.orchestra.run.smithi138.stdout: "jaeger_tracing_enable": "false", 2023-06-27T12:38:02.295 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_aio": "true", 2023-06-27T12:38:02.295 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_align_min_size": "65536", 2023-06-27T12:38:02.295 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_block_align": "true", 2023-06-27T12:38:02.295 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_block_size": "4096", 2023-06-27T12:38:02.295 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_dio": "true", 2023-06-27T12:38:02.295 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_discard": "false", 2023-06-27T12:38:02.296 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_force_aio": "false", 2023-06-27T12:38:02.296 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_ignore_corruption": "false", 2023-06-27T12:38:02.296 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_max_write_bytes": "10485760", 2023-06-27T12:38:02.296 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_max_write_entries": "100", 2023-06-27T12:38:02.296 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_replay_from": "0", 2023-06-27T12:38:02.296 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_throttle_high_multiple": "0.000000", 2023-06-27T12:38:02.296 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-06-27T12:38:02.296 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-06-27T12:38:02.297 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_throttle_max_multiple": "0.000000", 2023-06-27T12:38:02.297 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_write_header_frequency": "0", 2023-06-27T12:38:02.297 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_zero_on_create": "false", 2023-06-27T12:38:02.297 INFO:teuthology.orchestra.run.smithi138.stdout: "journaler_prefetch_periods": "10", 2023-06-27T12:38:02.297 INFO:teuthology.orchestra.run.smithi138.stdout: "journaler_prezero_periods": "5", 2023-06-27T12:38:02.297 INFO:teuthology.orchestra.run.smithi138.stdout: "journaler_write_head_interval": "15", 2023-06-27T12:38:02.297 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "", 2023-06-27T12:38:02.297 INFO:teuthology.orchestra.run.smithi138.stdout: "keyfile": "", 2023-06-27T12:38:02.298 INFO:teuthology.orchestra.run.smithi138.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-06-27T12:38:02.298 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_backend": "rocksdb", 2023-06-27T12:38:02.298 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_default_stripe_size": "65536", 2023-06-27T12:38:02.298 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_fsck_on_mount": "false", 2023-06-27T12:38:02.298 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_fsck_on_mount_deep": "true", 2023-06-27T12:38:02.298 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_max_bytes": "67108864", 2023-06-27T12:38:02.298 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_max_ops": "512", 2023-06-27T12:38:02.298 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_nid_prealloc": "1024", 2023-06-27T12:38:02.299 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_onode_map_size": "1024", 2023-06-27T12:38:02.299 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-06-27T12:38:02.299 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_sync_submit_transaction": "false", 2023-06-27T12:38:02.299 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_sync_transaction": "false", 2023-06-27T12:38:02.299 INFO:teuthology.orchestra.run.smithi138.stdout: "librados_thread_count": "2", 2023-06-27T12:38:02.299 INFO:teuthology.orchestra.run.smithi138.stdout: "lockdep": "false", 2023-06-27T12:38:02.299 INFO:teuthology.orchestra.run.smithi138.stdout: "lockdep_force_backtrace": "false", 2023-06-27T12:38:02.299 INFO:teuthology.orchestra.run.smithi138.stdout: "log_coarse_timestamps": "true", 2023-06-27T12:38:02.300 INFO:teuthology.orchestra.run.smithi138.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-06-27T12:38:02.300 INFO:teuthology.orchestra.run.smithi138.stdout: "log_flush_on_exit": "false", 2023-06-27T12:38:02.300 INFO:teuthology.orchestra.run.smithi138.stdout: "log_graylog_host": "127.0.0.1", 2023-06-27T12:38:02.300 INFO:teuthology.orchestra.run.smithi138.stdout: "log_graylog_port": "12201", 2023-06-27T12:38:02.300 INFO:teuthology.orchestra.run.smithi138.stdout: "log_max_new": "1000", 2023-06-27T12:38:02.300 INFO:teuthology.orchestra.run.smithi138.stdout: "log_max_recent": "10000", 2023-06-27T12:38:02.300 INFO:teuthology.orchestra.run.smithi138.stdout: "log_stderr_prefix": "", 2023-06-27T12:38:02.301 INFO:teuthology.orchestra.run.smithi138.stdout: "log_stop_at_utilization": "0.970000", 2023-06-27T12:38:02.301 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": "true", 2023-06-27T12:38:02.301 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_graylog": "false", 2023-06-27T12:38:02.301 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_journald": "false", 2023-06-27T12:38:02.301 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_stderr": "false", 2023-06-27T12:38:02.301 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_syslog": "false", 2023-06-27T12:38:02.301 INFO:teuthology.orchestra.run.smithi138.stdout: "max_rotating_auth_attempts": "10", 2023-06-27T12:38:02.301 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2023-06-27T12:38:02.302 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_action_on_write_error": "1", 2023-06-27T12:38:02.302 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_alternate_name_max": "8192", 2023-06-27T12:38:02.302 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_asio_thread_count": "2", 2023-06-27T12:38:02.302 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_export_pin": "true", 2023-06-27T12:38:02.302 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_fragment_dirs": "true", 2023-06-27T12:38:02.302 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-06-27T12:38:02.302 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_fragment_interval": "5", 2023-06-27T12:38:02.302 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_fragment_size_max": "100000", 2023-06-27T12:38:02.303 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_idle_threshold": "0.000000", 2023-06-27T12:38:02.303 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_interval": "10", 2023-06-27T12:38:02.303 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_max": "-1", 2023-06-27T12:38:02.303 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_max_until": "-1", 2023-06-27T12:38:02.303 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_merge_size": "50", 2023-06-27T12:38:02.303 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_midchunk": "0.300000", 2023-06-27T12:38:02.303 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_min_rebalance": "0.100000", 2023-06-27T12:38:02.303 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_min_start": "0.200000", 2023-06-27T12:38:02.304 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_minchunk": "0.001000", 2023-06-27T12:38:02.304 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_mode": "0", 2023-06-27T12:38:02.304 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_need_max": "1.200000", 2023-06-27T12:38:02.304 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_need_min": "0.800000", 2023-06-27T12:38:02.304 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-06-27T12:38:02.304 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_sample_interval": "3.000000", 2023-06-27T12:38:02.304 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_split_bits": "3", 2023-06-27T12:38:02.304 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_split_rd": "25000.000000", 2023-06-27T12:38:02.305 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_split_size": "10000", 2023-06-27T12:38:02.305 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_split_wr": "10000.000000", 2023-06-27T12:38:02.305 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_target_decay": "10.000000", 2023-06-27T12:38:02.305 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-06-27T12:38:02.305 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_beacon_grace": "15.000000", 2023-06-27T12:38:02.305 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_beacon_interval": "4.000000", 2023-06-27T12:38:02.305 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_beacon_mon_down_grace": "60", 2023-06-27T12:38:02.305 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_memory_limit": "4294967296", 2023-06-27T12:38:02.306 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_mid": "0.700000", 2023-06-27T12:38:02.306 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_release_free_interval": "10", 2023-06-27T12:38:02.306 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_reservation": "0.050000", 2023-06-27T12:38:02.306 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-06-27T12:38:02.306 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_trim_interval": "1", 2023-06-27T12:38:02.306 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_trim_threshold": "262144", 2023-06-27T12:38:02.306 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-06-27T12:38:02.307 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-06-27T12:38:02.307 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_client_delegate_inos_pct": "50", 2023-06-27T12:38:02.307 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_client_prealloc_inos": "1000", 2023-06-27T12:38:02.307 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-06-27T12:38:02.307 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_connect_bootstrapping": "false", 2023-06-27T12:38:02.307 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_damage_table_max_entries": "10000", 2023-06-27T12:38:02.307 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-06-27T12:38:02.307 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_debug_auth_pins": "false", 2023-06-27T12:38:02.308 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_debug_frag": "false", 2023-06-27T12:38:02.308 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_debug_scatterstat": "false", 2023-06-27T12:38:02.308 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_debug_subtrees": "false", 2023-06-27T12:38:02.308 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_decay_halflife": "5.000000", 2023-06-27T12:38:02.308 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_default_dir_hash": "2", 2023-06-27T12:38:02.308 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_defer_session_stale": "true", 2023-06-27T12:38:02.308 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_deny_all_reconnect": "false", 2023-06-27T12:38:02.308 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dir_keys_per_op": "16384", 2023-06-27T12:38:02.309 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dir_max_commit_size": "10", 2023-06-27T12:38:02.309 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dir_max_entries": "0", 2023-06-27T12:38:02.309 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dir_prefetch": "true", 2023-06-27T12:38:02.309 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dirstat_min_interval": "1.000000", 2023-06-27T12:38:02.309 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dump_cache_after_rejoin": "false", 2023-06-27T12:38:02.309 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dump_cache_on_map": "false", 2023-06-27T12:38:02.309 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dump_cache_threshold_file": "0", 2023-06-27T12:38:02.309 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-06-27T12:38:02.310 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_early_reply": "true", 2023-06-27T12:38:02.310 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_enable_op_tracker": "true", 2023-06-27T12:38:02.310 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_enforce_unique_name": "true", 2023-06-27T12:38:02.310 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_export_ephemeral_distributed": "true", 2023-06-27T12:38:02.310 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-06-27T12:38:02.310 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_export_ephemeral_random": "true", 2023-06-27T12:38:02.310 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-06-27T12:38:02.310 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-06-27T12:38:02.311 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_forward_all_requests_to_auth": "false", 2023-06-27T12:38:02.311 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-06-27T12:38:02.311 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-06-27T12:38:02.311 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_go_bad_corrupt_dentry": "true", 2023-06-27T12:38:02.311 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-06-27T12:38:02.311 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_health_cache_threshold": "1.500000", 2023-06-27T12:38:02.311 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_health_summarize_threshold": "10", 2023-06-27T12:38:02.311 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_heartbeat_grace": "15.000000", 2023-06-27T12:38:02.312 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_heartbeat_reset_grace": "1000", 2023-06-27T12:38:02.312 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_inject_health_dummy": "false", 2023-06-27T12:38:02.312 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2023-06-27T12:38:02.312 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_inject_migrator_session_race": "false", 2023-06-27T12:38:02.312 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2023-06-27T12:38:02.312 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_inject_skip_replaying_inotable": "false", 2023-06-27T12:38:02.312 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-06-27T12:38:02.312 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_join_fs": "", 2023-06-27T12:38:02.313 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_journal_format": "1", 2023-06-27T12:38:02.313 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_create_at": "0", 2023-06-27T12:38:02.313 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_export_at": "0", 2023-06-27T12:38:02.313 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_import_at": "0", 2023-06-27T12:38:02.313 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_journal_at": "0", 2023-06-27T12:38:02.313 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_journal_expire_at": "0", 2023-06-27T12:38:02.313 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_journal_replay_at": "0", 2023-06-27T12:38:02.313 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_link_at": "0", 2023-06-27T12:38:02.314 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_mdstable_at": "0", 2023-06-27T12:38:02.314 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_openc_at": "0", 2023-06-27T12:38:02.314 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_rename_at": "0", 2023-06-27T12:38:02.314 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_skip_replaying_inotable": "false", 2023-06-27T12:38:02.314 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_events_per_segment": "1024", 2023-06-27T12:38:02.314 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_max_events": "-1", 2023-06-27T12:38:02.314 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_max_segments": "128", 2023-06-27T12:38:02.315 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_pause": "false", 2023-06-27T12:38:02.315 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_segment_size": "0", 2023-06-27T12:38:02.315 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_skip_corrupt_events": "false", 2023-06-27T12:38:02.315 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_warn_factor": "2.000000", 2023-06-27T12:38:02.315 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_caps_per_client": "1048576", 2023-06-27T12:38:02.315 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_completed_flushes": "100000", 2023-06-27T12:38:02.315 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_completed_requests": "100000", 2023-06-27T12:38:02.315 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_export_size": "20971520", 2023-06-27T12:38:02.316 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_file_recover": "32", 2023-06-27T12:38:02.316 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_purge_files": "64", 2023-06-27T12:38:02.316 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_purge_ops": "8192", 2023-06-27T12:38:02.316 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-06-27T12:38:02.316 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-06-27T12:38:02.316 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_snaps_per_dir": "100", 2023-06-27T12:38:02.316 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_metrics_update_interval": "2", 2023-06-27T12:38:02.316 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_min_caps_per_client": "100", 2023-06-27T12:38:02.317 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_min_caps_working_set": "10000", 2023-06-27T12:38:02.317 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-06-27T12:38:02.317 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_numa_node": "-1", 2023-06-27T12:38:02.317 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-06-27T12:38:02.317 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_op_complaint_time": "30.000000", 2023-06-27T12:38:02.317 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_op_history_duration": "600", 2023-06-27T12:38:02.317 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_op_history_size": "20", 2023-06-27T12:38:02.317 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_op_log_threshold": "5", 2023-06-27T12:38:02.318 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_ping_grace": "15", 2023-06-27T12:38:02.318 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_ping_interval": "5", 2023-06-27T12:38:02.318 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-06-27T12:38:02.318 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-06-27T12:38:02.318 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_max_caps": "30000", 2023-06-27T12:38:02.318 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-06-27T12:38:02.318 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_max_decay_threshold": "131072", 2023-06-27T12:38:02.318 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-06-27T12:38:02.319 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_warning_threshold": "262144", 2023-06-27T12:38:02.319 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_reconnect_timeout": "45.000000", 2023-06-27T12:38:02.319 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_replay_interval": "1.000000", 2023-06-27T12:38:02.319 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-06-27T12:38:02.319 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-06-27T12:38:02.319 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_root_ino_gid": "0", 2023-06-27T12:38:02.319 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_root_ino_uid": "0", 2023-06-27T12:38:02.319 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-06-27T12:38:02.320 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_blocklist_on_evict": "true", 2023-06-27T12:38:02.320 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_blocklist_on_timeout": "true", 2023-06-27T12:38:02.320 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-06-27T12:38:02.320 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-06-27T12:38:02.320 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-06-27T12:38:02.320 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-06-27T12:38:02.320 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-06-27T12:38:02.320 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-06-27T12:38:02.321 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_shutdown_check": "0", 2023-06-27T12:38:02.321 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_skip_ino": "0", 2023-06-27T12:38:02.321 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sleep_rank_change": "0.000000", 2023-06-27T12:38:02.321 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_snap_max_uid": "4294967294", 2023-06-27T12:38:02.321 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_snap_min_uid": "0", 2023-06-27T12:38:02.321 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_snap_rstat": "false", 2023-06-27T12:38:02.321 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_standby_replay_damaged": "false", 2023-06-27T12:38:02.321 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_symlink_recovery": "true", 2023-06-27T12:38:02.322 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_task_status_update_interval": "2.000000", 2023-06-27T12:38:02.322 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_thrash_exports": "0", 2023-06-27T12:38:02.322 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_thrash_fragments": "0", 2023-06-27T12:38:02.322 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_tick_interval": "5.000000", 2023-06-27T12:38:02.322 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_valgrind_exit": "false", 2023-06-27T12:38:02.322 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_verify_backtrace": "1", 2023-06-27T12:38:02.322 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_verify_scatter": "false", 2023-06-27T12:38:02.323 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_wipe_ino_prealloc": "false", 2023-06-27T12:38:02.323 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_wipe_sessions": "false", 2023-06-27T12:38:02.323 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool_debug": "false", 2023-06-27T12:38:02.323 INFO:teuthology.orchestra.run.smithi138.stdout: "memstore_debug_omit_block_device_write": "false", 2023-06-27T12:38:02.323 INFO:teuthology.orchestra.run.smithi138.stdout: "memstore_device_bytes": "1073741824", 2023-06-27T12:38:02.323 INFO:teuthology.orchestra.run.smithi138.stdout: "memstore_page_set": "false", 2023-06-27T12:38:02.323 INFO:teuthology.orchestra.run.smithi138.stdout: "memstore_page_size": "65536", 2023-06-27T12:38:02.323 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_client_bytes": "134217728", 2023-06-27T12:38:02.324 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_client_messages": "512", 2023-06-27T12:38:02.324 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-06-27T12:38:02.324 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_connect_retry_interval": "1.000000", 2023-06-27T12:38:02.324 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-06-27T12:38:02.324 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-06-27T12:38:02.324 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_disabled_modules": "", 2023-06-27T12:38:02.324 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-06-27T12:38:02.324 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_max_pg_num_change": "128", 2023-06-27T12:38:02.325 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_mds_bytes": "134217728", 2023-06-27T12:38:02.325 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_mds_messages": "128", 2023-06-27T12:38:02.325 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-06-27T12:38:02.325 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_mon_bytes": "134217728", 2023-06-27T12:38:02.325 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_mon_messages": "128", 2023-06-27T12:38:02.325 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_osd_bytes": "536870912", 2023-06-27T12:38:02.325 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_osd_messages": "8192", 2023-06-27T12:38:02.325 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_pool": "true", 2023-06-27T12:38:02.326 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_service_beacon_grace": "60.000000", 2023-06-27T12:38:02.326 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_standby_modules": "true", 2023-06-27T12:38:02.326 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_stats_period": "5", 2023-06-27T12:38:02.326 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_stats_threshold": "5", 2023-06-27T12:38:02.326 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_tick_period": "2", 2023-06-27T12:38:02.326 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-06-27T12:38:02.326 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_accept_timeout_factor": "2.000000", 2023-06-27T12:38:02.326 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_allow_pool_delete": "true", 2023-06-27T12:38:02.327 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_allow_pool_size_one": "true", 2023-06-27T12:38:02.327 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_auth_validate_all_caps": "true", 2023-06-27T12:38:02.327 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-06-27T12:38:02.327 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-06-27T12:38:02.327 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_bytes": "104857600", 2023-06-27T12:38:02.327 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_directed_command_retry": "2", 2023-06-27T12:38:02.327 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_interval": "3.000000", 2023-06-27T12:38:02.328 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-06-27T12:38:02.328 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-06-27T12:38:02.328 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-06-27T12:38:02.328 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_parallel": "3", 2023-06-27T12:38:02.328 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_log_interval": "1.000000", 2023-06-27T12:38:02.328 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-06-27T12:38:02.328 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_ping_interval": "10.000000", 2023-06-27T12:38:02.328 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_ping_timeout": "30.000000", 2023-06-27T12:38:02.329 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_clock_drift_allowed": "1.000000", 2023-06-27T12:38:02.329 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-06-27T12:38:02.329 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-06-27T12:38:02.329 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_file_level": "debug", 2023-06-27T12:38:02.329 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_file": "true", 2023-06-27T12:38:02.329 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_graylog": "false", 2023-06-27T12:38:02.329 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-06-27T12:38:02.329 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-06-27T12:38:02.330 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_journald": "false", 2023-06-27T12:38:02.330 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_stderr": "false", 2023-06-27T12:38:02.330 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-06-27T12:38:02.330 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-06-27T12:38:02.330 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-06-27T12:38:02.330 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_compact_on_bootstrap": "false", 2023-06-27T12:38:02.330 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_compact_on_start": "false", 2023-06-27T12:38:02.331 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_compact_on_trim": "true", 2023-06-27T12:38:02.331 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_con_tracker_persist_interval": "10", 2023-06-27T12:38:02.331 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_con_tracker_score_halflife": "43200", 2023-06-27T12:38:02.331 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_config_key_max_entry_size": "65536", 2023-06-27T12:38:02.331 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cpu_threads": "4", 2023-06-27T12:38:02.331 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_crush_min_required_version": "hammer", 2023-06-27T12:38:02.331 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_daemon_bytes": "419430400", 2023-06-27T12:38:02.331 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-06-27T12:38:02.332 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_data_avail_crit": "5", 2023-06-27T12:38:02.332 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_data_avail_warn": "5", 2023-06-27T12:38:02.332 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_data_size_warn": "16106127360", 2023-06-27T12:38:02.332 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_block_osdmap_trim": "false", 2023-06-27T12:38:02.332 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-06-27T12:38:02.332 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_dump_json": "false", 2023-06-27T12:38:02.332 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-06-27T12:38:02.332 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_dump_transactions": "false", 2023-06-27T12:38:02.333 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_extra_checks": "false", 2023-06-27T12:38:02.333 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-06-27T12:38:02.333 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-06-27T12:38:02.333 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_no_require_quincy": "false", 2023-06-27T12:38:02.333 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_no_require_reef": "false", 2023-06-27T12:38:02.333 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-06-27T12:38:02.333 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_delta_reset_interval": "10.000000", 2023-06-27T12:38:02.333 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_dns_srv_name": "ceph-mon", 2023-06-27T12:38:02.334 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_down_mkfs_grace": "120", 2023-06-27T12:38:02.334 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_election_default_strategy": "1", 2023-06-27T12:38:02.334 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_election_timeout": "5.000000", 2023-06-27T12:38:02.334 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-06-27T12:38:02.334 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_elector_ping_divisor": "2", 2023-06-27T12:38:02.334 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_elector_ping_timeout": "2.000000", 2023-06-27T12:38:02.334 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_enable_op_tracker": "true", 2023-06-27T12:38:02.335 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_fake_pool_delete": "false", 2023-06-27T12:38:02.335 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_force_quorum_join": "false", 2023-06-27T12:38:02.335 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_globalid_prealloc": "10000", 2023-06-27T12:38:02.335 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_detail_to_clog": "false", 2023-06-27T12:38:02.335 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_log_update_period": "5", 2023-06-27T12:38:02.335 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_max_detail": "50", 2023-06-27T12:38:02.335 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_to_clog": "true", 2023-06-27T12:38:02.335 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_to_clog_interval": "600", 2023-06-27T12:38:02.336 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-06-27T12:38:02.336 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_host": "172.21.15.138", 2023-06-27T12:38:02.336 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_host_override": "", 2023-06-27T12:38:02.336 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_initial_members": "", 2023-06-27T12:38:02.336 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-06-27T12:38:02.336 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-06-27T12:38:02.336 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-06-27T12:38:02.336 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-06-27T12:38:02.337 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_keyvaluedb": "rocksdb", 2023-06-27T12:38:02.337 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_lease": "5.000000", 2023-06-27T12:38:02.337 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-06-27T12:38:02.337 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-06-27T12:38:02.337 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_log_full_interval": "50", 2023-06-27T12:38:02.337 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_log_max": "10000", 2023-06-27T12:38:02.337 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_log_max_summary": "50", 2023-06-27T12:38:02.337 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_log_entries_per_event": "4096", 2023-06-27T12:38:02.338 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_log_epochs": "500", 2023-06-27T12:38:02.338 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_mdsmap_epochs": "500", 2023-06-27T12:38:02.338 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_mgrmap_epochs": "500", 2023-06-27T12:38:02.338 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_osd": "10000", 2023-06-27T12:38:02.338 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_pg_per_osd": "10000", 2023-06-27T12:38:02.338 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_pool_pg_num": "65536", 2023-06-27T12:38:02.338 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-06-27T12:38:02.338 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-06-27T12:38:02.339 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mds_force_trim_to": "0", 2023-06-27T12:38:02.339 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mds_skip_sanity": "false", 2023-06-27T12:38:02.339 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_memory_autotune": "true", 2023-06-27T12:38:02.339 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_memory_target": "2147483648", 2023-06-27T12:38:02.339 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_beacon_grace": "30", 2023-06-27T12:38:02.339 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-06-27T12:38:02.339 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_digest_period": "5", 2023-06-27T12:38:02.339 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_inactive_grace": "60", 2023-06-27T12:38:02.340 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_mkfs_grace": "240", 2023-06-27T12:38:02.340 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-06-27T12:38:02.340 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_min_osdmap_epochs": "50", 2023-06-27T12:38:02.340 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_complaint_time": "30", 2023-06-27T12:38:02.340 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_history_duration": "600", 2023-06-27T12:38:02.340 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_history_size": "20", 2023-06-27T12:38:02.340 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_history_slow_op_size": "20", 2023-06-27T12:38:02.341 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_history_slow_op_threshold": "10", 2023-06-27T12:38:02.341 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_log_threshold": "5", 2023-06-27T12:38:02.341 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-06-27T12:38:02.341 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-06-27T12:38:02.341 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-06-27T12:38:02.341 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_auto_mark_in": "false", 2023-06-27T12:38:02.341 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_auto_mark_new_in": "true", 2023-06-27T12:38:02.341 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-06-27T12:38:02.342 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-06-27T12:38:02.342 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_cache_size": "500", 2023-06-27T12:38:02.342 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_cache_size_min": "134217728", 2023-06-27T12:38:02.342 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_crush_smoke_test": "true", 2023-06-27T12:38:02.342 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_destroyed_out_interval": "600", 2023-06-27T12:38:02.342 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_down_out_interval": "600", 2023-06-27T12:38:02.342 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-06-27T12:38:02.342 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-06-27T12:38:02.343 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_force_trim_to": "0", 2023-06-27T12:38:02.343 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_full_ratio": "0.950000", 2023-06-27T12:38:02.343 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-06-27T12:38:02.343 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_laggy_halflife": "3600", 2023-06-27T12:38:02.343 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_laggy_max_interval": "300", 2023-06-27T12:38:02.343 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_laggy_weight": "0.300000", 2023-06-27T12:38:02.343 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-06-27T12:38:02.344 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_max_creating_pgs": "1024", 2023-06-27T12:38:02.344 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_max_initial_pgs": "1024", 2023-06-27T12:38:02.344 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_min_down_reporters": "2", 2023-06-27T12:38:02.344 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-06-27T12:38:02.344 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-06-27T12:38:02.344 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-06-27T12:38:02.344 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_prime_pg_temp": "true", 2023-06-27T12:38:02.344 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-06-27T12:38:02.345 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-06-27T12:38:02.345 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_report_timeout": "900", 2023-06-27T12:38:02.345 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-06-27T12:38:02.345 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-06-27T12:38:02.345 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_warn_num_repaired": "10", 2023-06-27T12:38:02.345 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_warn_op_age": "32.000000", 2023-06-27T12:38:02.345 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-06-27T12:38:02.345 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osdmap_full_prune_interval": "2", 2023-06-27T12:38:02.346 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osdmap_full_prune_min": "15", 2023-06-27T12:38:02.346 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-06-27T12:38:02.346 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-06-27T12:38:02.346 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_stuck_threshold": "60", 2023-06-27T12:38:02.346 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-06-27T12:38:02.346 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_warn_min_objects": "10000", 2023-06-27T12:38:02.346 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_warn_min_per_osd": "0", 2023-06-27T12:38:02.346 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-06-27T12:38:02.347 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pool_quota_crit_threshold": "0", 2023-06-27T12:38:02.347 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pool_quota_warn_threshold": "0", 2023-06-27T12:38:02.347 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_probe_timeout": "2.000000", 2023-06-27T12:38:02.347 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_reweight_max_change": "0.050000", 2023-06-27T12:38:02.347 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_reweight_max_osds": "4", 2023-06-27T12:38:02.347 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-06-27T12:38:02.347 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-06-27T12:38:02.347 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-06-27T12:38:02.348 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-06-27T12:38:02.348 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-06-27T12:38:02.348 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_interval": "86400", 2023-06-27T12:38:02.348 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_max_keys": "100", 2023-06-27T12:38:02.348 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_timeout": "300", 2023-06-27T12:38:02.348 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_session_timeout": "300", 2023-06-27T12:38:02.348 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_smart_report_timeout": "5", 2023-06-27T12:38:02.349 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stat_smooth_intervals": "6", 2023-06-27T12:38:02.349 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-06-27T12:38:02.349 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stretch_pool_min_size": "2", 2023-06-27T12:38:02.349 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stretch_pool_size": "4", 2023-06-27T12:38:02.349 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-06-27T12:38:02.349 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_subscribe_interval": "86400.000000", 2023-06-27T12:38:02.349 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_debug": "false", 2023-06-27T12:38:02.349 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_max_payload_keys": "2000", 2023-06-27T12:38:02.350 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_max_payload_size": "1048576", 2023-06-27T12:38:02.350 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_provider_kill_at": "0", 2023-06-27T12:38:02.350 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_requester_kill_at": "0", 2023-06-27T12:38:02.350 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_timeout": "60.000000", 2023-06-27T12:38:02.350 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_target_pg_per_osd": "100", 2023-06-27T12:38:02.350 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_tick_interval": "5", 2023-06-27T12:38:02.350 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_timecheck_interval": "300.000000", 2023-06-27T12:38:02.350 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-06-27T12:38:02.351 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_use_min_delay_socket": "false", 2023-06-27T12:38:02.351 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_older_version_delay": "604800", 2023-06-27T12:38:02.351 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-06-27T12:38:02.351 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-06-27T12:38:02.351 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-06-27T12:38:02.351 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_filestore_osds": "false", 2023-06-27T12:38:02.351 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-06-27T12:38:02.351 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-06-27T12:38:02.352 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-06-27T12:38:02.352 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_misplaced": "false", 2023-06-27T12:38:02.352 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-06-27T12:38:02.352 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_older_version": "true", 2023-06-27T12:38:02.352 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-06-27T12:38:02.352 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_pool_no_app": "true", 2023-06-27T12:38:02.352 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-06-27T12:38:02.353 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-06-27T12:38:02.353 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-06-27T12:38:02.353 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-06-27T12:38:02.353 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_too_few_osds": "false", 2023-06-27T12:38:02.353 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-06-27T12:38:02.353 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-06-27T12:38:02.353 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": "", 2023-06-27T12:38:02.353 INFO:teuthology.orchestra.run.smithi138.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-06-27T12:38:02.354 INFO:teuthology.orchestra.run.smithi138.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2023-06-27T12:38:02.354 INFO:teuthology.orchestra.run.smithi138.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2023-06-27T12:38:02.354 INFO:teuthology.orchestra.run.smithi138.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-06-27T12:38:02.354 INFO:teuthology.orchestra.run.smithi138.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2023-06-27T12:38:02.354 INFO:teuthology.orchestra.run.smithi138.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2023-06-27T12:38:02.354 INFO:teuthology.orchestra.run.smithi138.stdout: "motr_tracing_enabled": "false", 2023-06-27T12:38:02.354 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_op_threads": "3", 2023-06-27T12:38:02.354 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_buffer_size": "131072", 2023-06-27T12:38:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_cm": "false", 2023-06-27T12:38:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_device_name": "", 2023-06-27T12:38:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_dscp": "96", 2023-06-27T12:38:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-06-27T12:38:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_gid_idx": "0", 2023-06-27T12:38:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_local_gid": "", 2023-06-27T12:38:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_polling_us": "1000", 2023-06-27T12:38:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_port_num": "1", 2023-06-27T12:38:02.356 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-06-27T12:38:02.356 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-06-27T12:38:02.356 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_roce_ver": "1", 2023-06-27T12:38:02.356 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_send_buffers": "1024", 2023-06-27T12:38:02.356 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_sl": "3", 2023-06-27T12:38:02.356 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_support_srq": "true", 2023-06-27T12:38:02.356 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_type": "ib", 2023-06-27T12:38:02.357 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_reap_threshold": "5", 2023-06-27T12:38:02.357 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_before_connect": "false", 2023-06-27T12:38:02.357 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_ipv4": "true", 2023-06-27T12:38:02.357 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_ipv6": "false", 2023-06-27T12:38:02.357 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_msgr1": "true", 2023-06-27T12:38:02.357 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_msgr2": "true", 2023-06-27T12:38:02.357 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_port_max": "7568", 2023-06-27T12:38:02.357 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_port_min": "6800", 2023-06-27T12:38:02.358 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_prefer_ipv4": "false", 2023-06-27T12:38:02.358 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_retry_count": "3", 2023-06-27T12:38:02.358 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_retry_delay": "5", 2023-06-27T12:38:02.358 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_client": "false", 2023-06-27T12:38:02.358 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_mds": "false", 2023-06-27T12:38:02.358 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_mgr": "false", 2023-06-27T12:38:02.358 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_mon": "false", 2023-06-27T12:38:02.358 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_osd": "false", 2023-06-27T12:38:02.359 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_client_mode": "crc", 2023-06-27T12:38:02.359 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_cluster_mode": "crc", 2023-06-27T12:38:02.359 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_cluster_type": "", 2023-06-27T12:38:02.359 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_compress_secure": "false", 2023-06-27T12:38:02.359 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_connection_idle_timeout": "900", 2023-06-27T12:38:02.359 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_connection_ready_timeout": "10", 2023-06-27T12:38:02.359 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_crc_data": "true", 2023-06-27T12:38:02.359 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_crc_header": "true", 2023-06-27T12:38:02.360 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_bad_msg": "false", 2023-06-27T12:38:02.360 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_bug": "true", 2023-06-27T12:38:02.360 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_old_message": "true", 2023-06-27T12:38:02.360 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_skipped_message": "false", 2023-06-27T12:38:02.360 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_unhandled_msg": "false", 2023-06-27T12:38:02.360 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-06-27T12:38:02.360 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_coremask": "0xF", 2023-06-27T12:38:02.360 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-06-27T12:38:02.361 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_devs_allowlist": "", 2023-06-27T12:38:02.361 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_enable_tso": "true", 2023-06-27T12:38:02.361 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-06-27T12:38:02.361 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-06-27T12:38:02.361 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_hugepages": "", 2023-06-27T12:38:02.361 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_hw_flow_control": "true", 2023-06-27T12:38:02.361 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-06-27T12:38:02.362 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_lro": "true", 2023-06-27T12:38:02.362 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_memory_channel": "4", 2023-06-27T12:38:02.362 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-06-27T12:38:02.362 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_pmd": "", 2023-06-27T12:38:02.362 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_port_id": "0", 2023-06-27T12:38:02.362 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-06-27T12:38:02.362 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dump_corrupt_message_level": "1", 2023-06-27T12:38:02.362 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dump_on_send": "false", 2023-06-27T12:38:02.363 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_initial_backoff": "0.200000", 2023-06-27T12:38:02.363 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_delay_max": "1.000000", 2023-06-27T12:38:02.363 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_delay_probability": "0.000000", 2023-06-27T12:38:02.363 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_delay_type": "", 2023-06-27T12:38:02.363 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_internal_delays": "0.000000", 2023-06-27T12:38:02.363 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_network_congestion": "0", 2023-06-27T12:38:02.363 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_socket_failures": "0", 2023-06-27T12:38:02.364 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_learn_addr_from_peer": "true", 2023-06-27T12:38:02.364 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_max_accept_failures": "4", 2023-06-27T12:38:02.364 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_max_backoff": "15.000000", 2023-06-27T12:38:02.364 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_mon_client_mode": "crc", 2023-06-27T12:38:02.364 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_mon_cluster_mode": "crc", 2023-06-27T12:38:02.364 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_mon_service_mode": "crc", 2023-06-27T12:38:02.364 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_osd_compress_min_size": "1024", 2023-06-27T12:38:02.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_osd_compress_mode": "none", 2023-06-27T12:38:02.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_osd_compression_algorithm": "snappy", 2023-06-27T12:38:02.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-06-27T12:38:02.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_pq_min_cost": "65536", 2023-06-27T12:38:02.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_public_type": "", 2023-06-27T12:38:02.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_service_mode": "crc", 2023-06-27T12:38:02.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_tcp_listen_backlog": "512", 2023-06-27T12:38:02.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_tcp_nodelay": "true", 2023-06-27T12:38:02.366 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-06-27T12:38:02.366 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_tcp_rcvbuf": "0", 2023-06-27T12:38:02.366 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_type": "async+posix", 2023-06-27T12:38:02.366 INFO:teuthology.orchestra.run.smithi138.stdout: "no_config_file": "false", 2023-06-27T12:38:02.366 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_completion_locks_per_session": "32", 2023-06-27T12:38:02.366 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_debug_inject_relock_delay": "false", 2023-06-27T12:38:02.366 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_inflight_op_bytes": "104857600", 2023-06-27T12:38:02.366 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_inflight_ops": "1024", 2023-06-27T12:38:02.367 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_inject_no_watch_ping": "false", 2023-06-27T12:38:02.367 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-06-27T12:38:02.367 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_tick_interval": "5.000000", 2023-06-27T12:38:02.367 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_timeout": "10.000000", 2023-06-27T12:38:02.367 INFO:teuthology.orchestra.run.smithi138.stdout: "objectstore_blackhole": "false", 2023-06-27T12:38:02.367 INFO:teuthology.orchestra.run.smithi138.stdout: "openssl_engine_opts": "", 2023-06-27T12:38:02.367 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_delay_time": "5.000000", 2023-06-27T12:38:02.367 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_hist_halflife": "1000", 2023-06-27T12:38:02.368 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_max_low_ops": "2", 2023-06-27T12:38:02.368 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_max_ops": "4", 2023-06-27T12:38:02.368 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_min_evict_effort": "0.100000", 2023-06-27T12:38:02.368 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_quantize_effort": "0.100000", 2023-06-27T12:38:02.368 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_slop": "0.020000", 2023-06-27T12:38:02.368 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_aggregated_slow_ops_logging": "true", 2023-06-27T12:38:02.368 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_allow_recovery_below_min_size": "true", 2023-06-27T12:38:02.369 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_asio_thread_count": "2", 2023-06-27T12:38:02.369 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_async_recovery_min_cost": "100", 2023-06-27T12:38:02.369 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_auto_mark_unfound_lost": "false", 2023-06-27T12:38:02.369 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backfill_retry_interval": "30.000000", 2023-06-27T12:38:02.369 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backfill_scan_max": "512", 2023-06-27T12:38:02.369 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backfill_scan_min": "64", 2023-06-27T12:38:02.369 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backoff_on_degraded": "false", 2023-06-27T12:38:02.369 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backoff_on_peering": "false", 2023-06-27T12:38:02.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backoff_on_unfound": "true", 2023-06-27T12:38:02.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_beacon_report_interval": "300", 2023-06-27T12:38:02.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bench_duration": "30", 2023-06-27T12:38:02.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bench_large_size_max_throughput": "104857600", 2023-06-27T12:38:02.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bench_max_block_size": "67108864", 2023-06-27T12:38:02.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bench_small_size_max_iops": "100", 2023-06-27T12:38:02.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_blkin_trace_all": "false", 2023-06-27T12:38:02.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_blocked_scrub_grace_period": "120", 2023-06-27T12:38:02.371 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2023-06-27T12:38:02.371 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2023-06-27T12:38:02.371 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2023-06-27T12:38:02.371 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_check_for_log_corruption": "false", 2023-06-27T12:38:02.371 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_check_max_object_name_len_on_startup": "true", 2023-06-27T12:38:02.371 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-06-27T12:38:02.371 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2023-06-27T12:38:02.371 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-06-27T12:38:02.372 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_class_update_on_start": "true", 2023-06-27T12:38:02.372 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_client_message_cap": "256", 2023-06-27T12:38:02.372 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_client_message_size_cap": "524288000", 2023-06-27T12:38:02.372 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_client_op_priority": "63", 2023-06-27T12:38:02.372 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_client_watch_timeout": "30", 2023-06-27T12:38:02.372 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_command_max_records": "256", 2023-06-27T12:38:02.372 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_compact_on_start": "false", 2023-06-27T12:38:02.373 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_copyfrom_max_chunk": "8388608", 2023-06-27T12:38:02.373 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_crush_chooseleaf_type": "0", 2023-06-27T12:38:02.373 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_crush_initial_weight": "-1.000000", 2023-06-27T12:38:02.373 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_crush_update_on_start": "true", 2023-06-27T12:38:02.373 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_crush_update_weight_set": "true", 2023-06-27T12:38:02.373 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2023-06-27T12:38:02.373 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2023-06-27T12:38:02.373 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2023-06-27T12:38:02.374 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_drop_ping_duration": "0", 2023-06-27T12:38:02.374 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_drop_ping_probability": "0.000000", 2023-06-27T12:38:02.374 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_feed_pullee": "-1", 2023-06-27T12:38:02.374 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_inject_copyfrom_error": "false", 2023-06-27T12:38:02.374 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2023-06-27T12:38:02.374 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2023-06-27T12:38:02.374 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_misdirected_ops": "false", 2023-06-27T12:38:02.374 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_no_acting_change": "false", 2023-06-27T12:38:02.375 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_no_purge_strays": "false", 2023-06-27T12:38:02.375 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_op_order": "false", 2023-06-27T12:38:02.375 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_pg_log_writeout": "false", 2023-06-27T12:38:02.375 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_pretend_recovery_active": "false", 2023-06-27T12:38:02.375 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_random_push_read_error": "0.000000", 2023-06-27T12:38:02.375 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2023-06-27T12:38:02.375 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_shutdown": "false", 2023-06-27T12:38:02.375 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2023-06-27T12:38:02.376 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2023-06-27T12:38:02.376 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_verify_cached_snaps": "false", 2023-06-27T12:38:02.376 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_verify_missing_on_start": "false", 2023-06-27T12:38:02.376 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_verify_snaps": "false", 2023-06-27T12:38:02.376 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_verify_stray_on_activate": "false", 2023-06-27T12:38:02.376 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_interval": "604800.000000", 2023-06-27T12:38:02.376 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_keys": "1024", 2023-06-27T12:38:02.376 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2023-06-27T12:38:02.377 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2023-06-27T12:38:02.377 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2023-06-27T12:38:02.377 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_stride": "524288", 2023-06-27T12:38:02.377 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2023-06-27T12:38:02.377 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_default_data_pool_replay_window": "5", 2023-06-27T12:38:02.377 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_default_notify_timeout": "30", 2023-06-27T12:38:02.377 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_delete_sleep": "0.000000", 2023-06-27T12:38:02.378 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_delete_sleep_hdd": "5.000000", 2023-06-27T12:38:02.378 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_delete_sleep_hybrid": "1.000000", 2023-06-27T12:38:02.378 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_delete_sleep_ssd": "1.000000", 2023-06-27T12:38:02.378 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_discard_disconnected_ops": "true", 2023-06-27T12:38:02.378 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_enable_op_tracker": "true", 2023-06-27T12:38:02.378 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2023-06-27T12:38:02.378 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_extblkdev_plugins": "vdo", 2023-06-27T12:38:02.378 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_failsafe_full_ratio": "0.970000", 2023-06-27T12:38:02.379 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_fast_fail_on_connection_refused": "true", 2023-06-27T12:38:02.379 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_fast_info": "true", 2023-06-27T12:38:02.379 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_fast_shutdown": "true", 2023-06-27T12:38:02.379 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_fast_shutdown_notify_mon": "true", 2023-06-27T12:38:02.379 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_fast_shutdown_timeout": "15", 2023-06-27T12:38:02.379 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_find_best_info_ignore_history_les": "false", 2023-06-27T12:38:02.379 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_force_auth_primary_missing_objects": "100", 2023-06-27T12:38:02.379 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2023-06-27T12:38:02.380 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_function_tracing": "false", 2023-06-27T12:38:02.380 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_grace": "20", 2023-06-27T12:38:02.380 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_interval": "6", 2023-06-27T12:38:02.380 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2023-06-27T12:38:02.380 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_min_peers": "10", 2023-06-27T12:38:02.380 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_min_size": "2000", 2023-06-27T12:38:02.380 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_stale": "600", 2023-06-27T12:38:02.380 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2023-06-27T12:38:02.381 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_hit_set_max_size": "100000", 2023-06-27T12:38:02.381 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_hit_set_min_size": "1000", 2023-06-27T12:38:02.381 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_hit_set_namespace": ".ceph-internal", 2023-06-27T12:38:02.381 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_ignore_stale_divergent_priors": "false", 2023-06-27T12:38:02.381 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2023-06-27T12:38:02.381 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_inject_failure_on_pg_removal": "false", 2023-06-27T12:38:02.381 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2023-06-27T12:38:02.381 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_journal_flush_on_shutdown": "true", 2023-06-27T12:38:02.382 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_journal_size": "5120", 2023-06-27T12:38:02.382 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_kill_backfill_at": "0", 2023-06-27T12:38:02.382 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_loop_before_reset_tphandle": "64", 2023-06-27T12:38:02.382 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_cache_size": "50", 2023-06-27T12:38:02.382 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_dedup": "true", 2023-06-27T12:38:02.382 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_message_max": "40", 2023-06-27T12:38:02.382 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_message_max_bytes": "10485760", 2023-06-27T12:38:02.382 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_share_max_epochs": "40", 2023-06-27T12:38:02.383 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_attr_name_len": "100", 2023-06-27T12:38:02.383 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_attr_size": "0", 2023-06-27T12:38:02.383 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_backfills": "1", 2023-06-27T12:38:02.383 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_markdown_count": "5", 2023-06-27T12:38:02.383 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_markdown_period": "600", 2023-06-27T12:38:02.383 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_object_name_len": "2048", 2023-06-27T12:38:02.383 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_object_namespace_len": "256", 2023-06-27T12:38:02.384 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_object_size": "134217728", 2023-06-27T12:38:02.384 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2023-06-27T12:38:02.384 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_omap_entries_per_request": "1024", 2023-06-27T12:38:02.384 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_pg_blocked_by": "16", 2023-06-27T12:38:02.384 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_pg_log_entries": "10000", 2023-06-27T12:38:02.384 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2023-06-27T12:38:02.384 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_pgls": "1024", 2023-06-27T12:38:02.384 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_push_cost": "8388608", 2023-06-27T12:38:02.385 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_push_objects": "10", 2023-06-27T12:38:02.385 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_scrubs": "3", 2023-06-27T12:38:02.385 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2023-06-27T12:38:02.385 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_trimming_pgs": "2", 2023-06-27T12:38:02.385 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_write_op_reply_len": "64", 2023-06-27T12:38:02.385 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_write_size": "90", 2023-06-27T12:38:02.385 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2023-06-27T12:38:02.385 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2023-06-27T12:38:02.386 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2023-06-27T12:38:02.386 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2023-06-27T12:38:02.386 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2023-06-27T12:38:02.386 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2023-06-27T12:38:02.386 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2023-06-27T12:38:02.386 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_override_recovery_settings": "false", 2023-06-27T12:38:02.386 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_profile": "balanced", 2023-06-27T12:38:02.386 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2023-06-27T12:38:02.387 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2023-06-27T12:38:02.387 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2023-06-27T12:38:02.387 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2023-06-27T12:38:02.387 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2023-06-27T12:38:02.387 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2023-06-27T12:38:02.387 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2023-06-27T12:38:02.387 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2023-06-27T12:38:02.387 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2023-06-27T12:38:02.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_client_wgt": "1", 2023-06-27T12:38:02.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_skip_benchmark": "false", 2023-06-27T12:38:02.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_base": "805306368", 2023-06-27T12:38:02.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_cache_min": "134217728", 2023-06-27T12:38:02.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_cache_resize_interval": "1.000000", 2023-06-27T12:38:02.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_expected_fragmentation": "0.150000", 2023-06-27T12:38:02.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_target": "4294967296", 2023-06-27T12:38:02.389 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_target_autotune": "false", 2023-06-27T12:38:02.389 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2023-06-27T12:38:02.389 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_min_pg_log_entries": "250", 2023-06-27T12:38:02.389 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_min_recovery_priority": "0", 2023-06-27T12:38:02.389 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_heartbeat_interval": "30", 2023-06-27T12:38:02.389 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2023-06-27T12:38:02.389 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_report_interval": "5", 2023-06-27T12:38:02.389 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_report_max_in_flight": "2", 2023-06-27T12:38:02.390 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_shutdown_timeout": "5.000000", 2023-06-27T12:38:02.390 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_num_cache_shards": "32", 2023-06-27T12:38:02.390 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_num_op_tracker_shard": "32", 2023-06-27T12:38:02.390 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_numa_auto_affinity": "true", 2023-06-27T12:38:02.390 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_numa_node": "-1", 2023-06-27T12:38:02.390 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_numa_prefer_iface": "true", 2023-06-27T12:38:02.390 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_object_clean_region_max_num_intervals": "10", 2023-06-27T12:38:02.390 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_objecter_finishers": "1", 2023-06-27T12:38:02.391 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_objectstore": "bluestore", 2023-06-27T12:38:02.391 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_objectstore_fuse": "false", 2023-06-27T12:38:02.391 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_objectstore_tracing": "false", 2023-06-27T12:38:02.391 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_complaint_time": "30.000000", 2023-06-27T12:38:02.391 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_history_duration": "600", 2023-06-27T12:38:02.391 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_history_size": "20", 2023-06-27T12:38:02.391 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_history_slow_op_size": "20", 2023-06-27T12:38:02.391 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2023-06-27T12:38:02.392 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_log_threshold": "5", 2023-06-27T12:38:02.392 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_shards": "0", 2023-06-27T12:38:02.392 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_shards_hdd": "5", 2023-06-27T12:38:02.392 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_shards_ssd": "8", 2023-06-27T12:38:02.392 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_threads_per_shard": "0", 2023-06-27T12:38:02.392 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2023-06-27T12:38:02.392 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2023-06-27T12:38:02.392 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2023-06-27T12:38:02.393 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_pq_min_cost": "65536", 2023-06-27T12:38:02.393 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_queue": "mclock_scheduler", 2023-06-27T12:38:02.393 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_queue_cut_off": "high", 2023-06-27T12:38:02.393 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_thread_suicide_timeout": "150", 2023-06-27T12:38:02.393 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_thread_timeout": "15", 2023-06-27T12:38:02.393 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_open_classes_on_start": "true", 2023-06-27T12:38:02.393 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_os_flags": "0", 2023-06-27T12:38:02.393 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_peering_op_priority": "255", 2023-06-27T12:38:02.394 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_delete_cost": "1048576", 2023-06-27T12:38:02.394 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_delete_priority": "5", 2023-06-27T12:38:02.394 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2023-06-27T12:38:02.394 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2023-06-27T12:38:02.394 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_log_dups_tracked": "3000", 2023-06-27T12:38:02.394 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_log_trim_max": "10000", 2023-06-27T12:38:02.394 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_log_trim_min": "100", 2023-06-27T12:38:02.395 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2023-06-27T12:38:02.395 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_object_context_cache_count": "64", 2023-06-27T12:38:02.395 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_stat_report_interval_max": "500", 2023-06-27T12:38:02.395 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2023-06-27T12:38:02.395 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_min_evict_age": "0", 2023-06-27T12:38:02.395 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_min_flush_age": "0", 2023-06-27T12:38:02.395 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2023-06-27T12:38:02.395 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2023-06-27T12:38:02.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2023-06-27T12:38:02.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_crimson": "true", 2023-06-27T12:38:02.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_crush_rule": "-1", 2023-06-27T12:38:02.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_ec_fast_read": "false", 2023-06-27T12:38:02.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2023-06-27T12:38:02.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flag_bulk": "false", 2023-06-27T12:38:02.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flag_hashpspool": "true", 2023-06-27T12:38:02.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flag_nodelete": "false", 2023-06-27T12:38:02.397 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flag_nopgchange": "false", 2023-06-27T12:38:02.397 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flag_nosizechange": "false", 2023-06-27T12:38:02.397 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flags": "0", 2023-06-27T12:38:02.397 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2023-06-27T12:38:02.397 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_min_size": "2", 2023-06-27T12:38:02.397 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2023-06-27T12:38:02.397 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_pg_num": "32", 2023-06-27T12:38:02.397 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_pgp_num": "0", 2023-06-27T12:38:02.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2023-06-27T12:38:02.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_size": "3", 2023-06-27T12:38:02.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_type": "replicated", 2023-06-27T12:38:02.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2023-06-27T12:38:02.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_use_gmt_hitset": "true", 2023-06-27T12:38:02.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_push_per_object_cost": "1000", 2023-06-27T12:38:02.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_read_ec_check_for_errors": "false", 2023-06-27T12:38:02.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recover_clone_overlap": "true", 2023-06-27T12:38:02.399 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recover_clone_overlap_limit": "10", 2023-06-27T12:38:02.399 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_cost": "20971520", 2023-06-27T12:38:02.399 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_delay_start": "0.000000", 2023-06-27T12:38:02.399 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_active": "0", 2023-06-27T12:38:02.399 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_active_hdd": "3", 2023-06-27T12:38:02.399 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_active_ssd": "10", 2023-06-27T12:38:02.399 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_chunk": "8388608", 2023-06-27T12:38:02.399 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2023-06-27T12:38:02.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_single_start": "1", 2023-06-27T12:38:02.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_op_priority": "3", 2023-06-27T12:38:02.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_op_warn_multiple": "16", 2023-06-27T12:38:02.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_priority": "5", 2023-06-27T12:38:02.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_retry_interval": "30.000000", 2023-06-27T12:38:02.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_sleep": "0.000000", 2023-06-27T12:38:02.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_sleep_hdd": "0.100000", 2023-06-27T12:38:02.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2023-06-27T12:38:02.401 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_sleep_ssd": "0.000000", 2023-06-27T12:38:02.401 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_repair_during_recovery": "false", 2023-06-27T12:38:02.401 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_requested_scrub_priority": "120", 2023-06-27T12:38:02.401 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2023-06-27T12:38:02.401 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_rollback_to_cluster_snap": "", 2023-06-27T12:38:02.401 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_auto_repair": "false", 2023-06-27T12:38:02.401 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_auto_repair_num_errors": "5", 2023-06-27T12:38:02.401 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_backoff_ratio": "0.660000", 2023-06-27T12:38:02.402 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_begin_hour": "0", 2023-06-27T12:38:02.402 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_begin_week_day": "0", 2023-06-27T12:38:02.402 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_chunk_max": "25", 2023-06-27T12:38:02.402 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_chunk_min": "5", 2023-06-27T12:38:02.402 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_cost": "52428800", 2023-06-27T12:38:02.402 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_during_recovery": "false", 2023-06-27T12:38:02.402 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_end_hour": "0", 2023-06-27T12:38:02.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_end_week_day": "0", 2023-06-27T12:38:02.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_event_cost": "4096", 2023-06-27T12:38:02.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_extended_sleep": "0.000000", 2023-06-27T12:38:02.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2023-06-27T12:38:02.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_invalid_stats": "true", 2023-06-27T12:38:02.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_load_threshold": "0.500000", 2023-06-27T12:38:02.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_max_interval": "604800.000000", 2023-06-27T12:38:02.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_max_preemptions": "5", 2023-06-27T12:38:02.404 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_min_interval": "86400.000000", 2023-06-27T12:38:02.404 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_priority": "5", 2023-06-27T12:38:02.404 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_reservation_timeout": "5000", 2023-06-27T12:38:02.404 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_sleep": "0.000000", 2023-06-27T12:38:02.404 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_slow_reservation_response": "2200", 2023-06-27T12:38:02.404 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_shallow_scrub_chunk_max": "100", 2023-06-27T12:38:02.404 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_shallow_scrub_chunk_min": "50", 2023-06-27T12:38:02.404 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_shutdown_pgref_assert": "false", 2023-06-27T12:38:02.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_skip_data_digest": "false", 2023-06-27T12:38:02.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_smart_report_timeout": "5", 2023-06-27T12:38:02.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_cost": "1048576", 2023-06-27T12:38:02.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_priority": "5", 2023-06-27T12:38:02.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_sleep": "0.000000", 2023-06-27T12:38:02.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2023-06-27T12:38:02.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2023-06-27T12:38:02.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2023-06-27T12:38:02.406 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_stats_update_period_not_scrubbing": "120", 2023-06-27T12:38:02.406 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_stats_update_period_scrubbing": "15", 2023-06-27T12:38:02.406 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2023-06-27T12:38:02.406 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_target_transaction_size": "30", 2023-06-27T12:38:02.406 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_count": "4", 2023-06-27T12:38:02.406 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2023-06-27T12:38:02.406 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2023-06-27T12:38:02.406 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2023-06-27T12:38:02.407 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2023-06-27T12:38:02.407 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2023-06-27T12:38:02.407 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2023-06-27T12:38:02.407 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_mode": "writeback", 2023-06-27T12:38:02.407 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2023-06-27T12:38:02.407 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_promote_max_objects_sec": "25", 2023-06-27T12:38:02.407 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tracing": "false", 2023-06-27T12:38:02.407 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_use_stale_snap": "false", 2023-06-27T12:38:02.408 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2023-06-27T12:38:02.408 INFO:teuthology.orchestra.run.smithi138.stdout: "osdc_blkin_trace_all": "false", 2023-06-27T12:38:02.408 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_kill_at": "0", 2023-06-27T12:38:02.408 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_max_join_drift": "10", 2023-06-27T12:38:02.408 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_min": "500", 2023-06-27T12:38:02.408 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_min_wait": "0.050000", 2023-06-27T12:38:02.408 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_propose_interval": "1.000000", 2023-06-27T12:38:02.409 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_service_trim_max": "500", 2023-06-27T12:38:02.409 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_service_trim_max_multiplier": "20", 2023-06-27T12:38:02.409 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_service_trim_min": "10", 2023-06-27T12:38:02.409 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_stash_full_interval": "25", 2023-06-27T12:38:02.409 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_trim_max": "500", 2023-06-27T12:38:02.409 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_trim_min": "250", 2023-06-27T12:38:02.409 INFO:teuthology.orchestra.run.smithi138.stdout: "perf": "true", 2023-06-27T12:38:02.409 INFO:teuthology.orchestra.run.smithi138.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2023-06-27T12:38:02.410 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin_crypto_accelerator": "crypto_isal", 2023-06-27T12:38:02.410 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin_dir": "/usr/lib64/ceph", 2023-06-27T12:38:02.410 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "-", 2023-06-27T12:38:02.410 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrv": "", 2023-06-27T12:38:02.410 INFO:teuthology.orchestra.run.smithi138.stdout: "public_bind_addr": "-", 2023-06-27T12:38:02.410 INFO:teuthology.orchestra.run.smithi138.stdout: "public_network": "", 2023-06-27T12:38:02.410 INFO:teuthology.orchestra.run.smithi138.stdout: "public_network_interface": "", 2023-06-27T12:38:02.410 INFO:teuthology.orchestra.run.smithi138.stdout: "qat_compressor_enabled": "false", 2023-06-27T12:38:02.411 INFO:teuthology.orchestra.run.smithi138.stdout: "qat_compressor_session_max_number": "256", 2023-06-27T12:38:02.411 INFO:teuthology.orchestra.run.smithi138.stdout: "rados_mon_op_timeout": "0", 2023-06-27T12:38:02.411 INFO:teuthology.orchestra.run.smithi138.stdout: "rados_osd_op_timeout": "0", 2023-06-27T12:38:02.411 INFO:teuthology.orchestra.run.smithi138.stdout: "rados_tracing": "false", 2023-06-27T12:38:02.411 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_atime_update_interval": "60", 2023-06-27T12:38:02.411 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2023-06-27T12:38:02.411 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_balance_parent_reads": "false", 2023-06-27T12:38:02.411 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_balance_snap_reads": "false", 2023-06-27T12:38:02.412 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_blkin_trace_all": "false", 2023-06-27T12:38:02.412 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_blocklist_expire_seconds": "0", 2023-06-27T12:38:02.412 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_blocklist_on_break_lock": "true", 2023-06-27T12:38:02.412 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache": "true", 2023-06-27T12:38:02.412 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_block_writes_upfront": "false", 2023-06-27T12:38:02.412 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_max_dirty": "25165824", 2023-06-27T12:38:02.412 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_max_dirty_age": "1.000000", 2023-06-27T12:38:02.413 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_max_dirty_object": "0", 2023-06-27T12:38:02.413 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_policy": "writearound", 2023-06-27T12:38:02.413 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_size": "33554432", 2023-06-27T12:38:02.413 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_target_dirty": "16777216", 2023-06-27T12:38:02.413 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_writethrough_until_flush": "true", 2023-06-27T12:38:02.413 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_clone_copy_on_read": "false", 2023-06-27T12:38:02.413 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_compression_hint": "none", 2023-06-27T12:38:02.413 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_concurrent_management_ops": "10", 2023-06-27T12:38:02.414 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_config_pool_override_update_timestamp": "0", 2023-06-27T12:38:02.414 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_clone_format": "auto", 2023-06-27T12:38:02.414 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_data_pool": "", 2023-06-27T12:38:02.414 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_features": "61", 2023-06-27T12:38:02.414 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_format": "2", 2023-06-27T12:38:02.414 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_map_options": "", 2023-06-27T12:38:02.414 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_order": "22", 2023-06-27T12:38:02.414 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_pool": "rbd", 2023-06-27T12:38:02.415 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2023-06-27T12:38:02.415 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_stripe_count": "0", 2023-06-27T12:38:02.415 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_stripe_unit": "0", 2023-06-27T12:38:02.415 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_disable_zero_copy_writes": "true", 2023-06-27T12:38:02.415 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_discard_granularity_bytes": "65536", 2023-06-27T12:38:02.415 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_discard_on_zeroed_write_same": "true", 2023-06-27T12:38:02.415 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_enable_alloc_hint": "true", 2023-06-27T12:38:02.415 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2023-06-27T12:38:02.416 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_io_scheduler": "simple", 2023-06-27T12:38:02.416 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2023-06-27T12:38:02.416 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_commit_age": "5.000000", 2023-06-27T12:38:02.416 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2023-06-27T12:38:02.416 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_max_payload_bytes": "16384", 2023-06-27T12:38:02.416 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_flush_age": "0.000000", 2023-06-27T12:38:02.416 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_flush_bytes": "1048576", 2023-06-27T12:38:02.416 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_flush_interval": "0", 2023-06-27T12:38:02.417 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2023-06-27T12:38:02.417 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2023-06-27T12:38:02.417 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_order": "24", 2023-06-27T12:38:02.417 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_pool": "", 2023-06-27T12:38:02.417 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_splay_width": "4", 2023-06-27T12:38:02.417 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_localize_parent_reads": "false", 2023-06-27T12:38:02.417 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_localize_snap_reads": "false", 2023-06-27T12:38:02.417 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2023-06-27T12:38:02.418 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2023-06-27T12:38:02.418 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2023-06-27T12:38:02.418 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2023-06-27T12:38:02.418 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2023-06-27T12:38:02.418 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2023-06-27T12:38:02.418 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_policy_type": "simple", 2023-06-27T12:38:02.418 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2023-06-27T12:38:02.419 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_state_check_interval": "30", 2023-06-27T12:38:02.419 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2023-06-27T12:38:02.419 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2023-06-27T12:38:02.419 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2023-06-27T12:38:02.419 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2023-06-27T12:38:02.419 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2023-06-27T12:38:02.419 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_autotune": "true", 2023-06-27T12:38:02.419 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_base": "805306368", 2023-06-27T12:38:02.420 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2023-06-27T12:38:02.420 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_cache_min": "134217728", 2023-06-27T12:38:02.420 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2023-06-27T12:38:02.420 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2023-06-27T12:38:02.420 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_target": "4294967296", 2023-06-27T12:38:02.420 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_perf_stats_prio": "5", 2023-06-27T12:38:02.420 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2023-06-27T12:38:02.420 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2023-06-27T12:38:02.421 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirroring_delete_delay": "0", 2023-06-27T12:38:02.421 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2023-06-27T12:38:02.421 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirroring_replay_delay": "0", 2023-06-27T12:38:02.421 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2023-06-27T12:38:02.421 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2023-06-27T12:38:02.421 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_move_to_trash_on_remove": "false", 2023-06-27T12:38:02.421 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2023-06-27T12:38:02.421 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mtime_update_interval": "60", 2023-06-27T12:38:02.422 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_non_blocking_aio": "true", 2023-06-27T12:38:02.422 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_op_thread_timeout": "60", 2023-06-27T12:38:02.422 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_op_threads": "1", 2023-06-27T12:38:02.422 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_parent_cache_enabled": "false", 2023-06-27T12:38:02.422 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_persistent_cache_mode": "disabled", 2023-06-27T12:38:02.422 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_persistent_cache_path": "/tmp", 2023-06-27T12:38:02.422 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_persistent_cache_size": "1073741824", 2023-06-27T12:38:02.422 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_plugins": "", 2023-06-27T12:38:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_bps_burst": "0", 2023-06-27T12:38:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_bps_burst_seconds": "1", 2023-06-27T12:38:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_bps_limit": "0", 2023-06-27T12:38:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_exclude_ops": "0", 2023-06-27T12:38:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_iops_burst": "0", 2023-06-27T12:38:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_iops_burst_seconds": "1", 2023-06-27T12:38:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_iops_limit": "0", 2023-06-27T12:38:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_bps_burst": "0", 2023-06-27T12:38:02.424 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2023-06-27T12:38:02.424 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_bps_limit": "0", 2023-06-27T12:38:02.424 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_iops_burst": "0", 2023-06-27T12:38:02.424 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2023-06-27T12:38:02.424 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_iops_limit": "0", 2023-06-27T12:38:02.424 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_schedule_tick_min": "50", 2023-06-27T12:38:02.424 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_bps_burst": "0", 2023-06-27T12:38:02.424 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2023-06-27T12:38:02.425 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_bps_limit": "0", 2023-06-27T12:38:02.425 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_iops_burst": "0", 2023-06-27T12:38:02.425 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2023-06-27T12:38:02.425 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_iops_limit": "0", 2023-06-27T12:38:02.425 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_quiesce_notification_attempts": "10", 2023-06-27T12:38:02.425 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_read_from_replica_policy": "default", 2023-06-27T12:38:02.425 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2023-06-27T12:38:02.425 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_readahead_max_bytes": "524288", 2023-06-27T12:38:02.426 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_readahead_trigger_requests": "10", 2023-06-27T12:38:02.426 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_request_timed_out_seconds": "30", 2023-06-27T12:38:02.426 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_skip_partial_discard": "true", 2023-06-27T12:38:02.426 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2023-06-27T12:38:02.426 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_tracing": "false", 2023-06-27T12:38:02.426 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_validate_names": "true", 2023-06-27T12:38:02.426 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_validate_pool": "true", 2023-06-27T12:38:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: "restapi_base_url": "", 2023-06-27T12:38:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: "restapi_log_level": "", 2023-06-27T12:38:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_acl_grants_max_num": "100", 2023-06-27T12:38:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_admin_entry": "admin", 2023-06-27T12:38:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2023-06-27T12:38:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_backend_store": "rados", 2023-06-27T12:38:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_barbican_url": "", 2023-06-27T12:38:02.427 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_beast_enable_async": "true", 2023-06-27T12:38:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2023-06-27T12:38:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_default_quota_max_size": "-1", 2023-06-27T12:38:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_index_max_aio": "128", 2023-06-27T12:38:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2023-06-27T12:38:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_quota_cache_size": "10000", 2023-06-27T12:38:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_quota_ttl": "600", 2023-06-27T12:38:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_sync_spawn_window": "20", 2023-06-27T12:38:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cache_enabled": "true", 2023-06-27T12:38:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cache_expiry_interval": "900", 2023-06-27T12:38:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cache_lru_size": "10000", 2023-06-27T12:38:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_config_store": "rados", 2023-06-27T12:38:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_content_length_compat": "false", 2023-06-27T12:38:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_copy_obj_progress": "true", 2023-06-27T12:38:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2023-06-27T12:38:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cors_rules_max_num": "100", 2023-06-27T12:38:02.429 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cross_domain_policy": "", 2023-06-27T12:38:02.430 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_default_encryption_key": "", 2023-06-27T12:38:02.430 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_kmip_addr": "", 2023-06-27T12:38:02.430 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_kmip_ca_path": "", 2023-06-27T12:38:02.430 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_kmip_client_cert": "", 2023-06-27T12:38:02.430 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_kmip_client_key": "", 2023-06-27T12:38:02.430 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_kmip_kms_key_template": "", 2023-06-27T12:38:02.430 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_kmip_password": "", 2023-06-27T12:38:02.430 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2023-06-27T12:38:02.431 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_kmip_username": "", 2023-06-27T12:38:02.431 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_require_ssl": "true", 2023-06-27T12:38:02.431 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2023-06-27T12:38:02.431 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2023-06-27T12:38:02.431 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_backend": "vault", 2023-06-27T12:38:02.431 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2023-06-27T12:38:02.431 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2023-06-27T12:38:02.431 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2023-06-27T12:38:02.432 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2023-06-27T12:38:02.432 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2023-06-27T12:38:02.432 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2023-06-27T12:38:02.432 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2023-06-27T12:38:02.432 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2023-06-27T12:38:02.432 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2023-06-27T12:38:02.432 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2023-06-27T12:38:02.433 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2023-06-27T12:38:02.433 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_suppress_logs": "true", 2023-06-27T12:38:02.433 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_addr": "", 2023-06-27T12:38:02.433 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_auth": "token", 2023-06-27T12:38:02.433 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_namespace": "", 2023-06-27T12:38:02.433 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_prefix": "", 2023-06-27T12:38:02.433 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_secret_engine": "transit", 2023-06-27T12:38:02.433 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_ssl_cacert": "", 2023-06-27T12:38:02.434 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2023-06-27T12:38:02.434 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2023-06-27T12:38:02.434 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_token_file": "", 2023-06-27T12:38:02.434 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_verify_ssl": "true", 2023-06-27T12:38:02.434 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_curl_buffersize": "524288", 2023-06-27T12:38:02.434 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_curl_low_speed_limit": "1024", 2023-06-27T12:38:02.434 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_curl_low_speed_time": "300", 2023-06-27T12:38:02.434 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_curl_tcp_keepalive": "0", 2023-06-27T12:38:02.435 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_curl_wait_timeout_ms": "1000", 2023-06-27T12:38:02.435 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2023-06-27T12:38:02.435 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2023-06-27T12:38:02.435 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2023-06-27T12:38:02.435 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2023-06-27T12:38:02.435 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d3n_l1_fadvise": "4", 2023-06-27T12:38:02.435 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2023-06-27T12:38:02.435 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d3n_libaio_aio_num": "64", 2023-06-27T12:38:02.436 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d3n_libaio_aio_threads": "20", 2023-06-27T12:38:02.436 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d4n_host": "127.0.0.1", 2023-06-27T12:38:02.436 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_d4n_port": "6379", 2023-06-27T12:38:02.436 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2023-06-27T12:38:02.436 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_log_changes_size": "1000", 2023-06-27T12:38:02.436 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_log_num_shards": "128", 2023-06-27T12:38:02.436 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_log_obj_prefix": "data_log", 2023-06-27T12:38:02.436 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_log_window": "30", 2023-06-27T12:38:02.437 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_notify_interval_msec": "0", 2023-06-27T12:38:02.437 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_sync_poll_interval": "20", 2023-06-27T12:38:02.437 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_sync_spawn_window": "20", 2023-06-27T12:38:02.437 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_default_data_log_backing": "fifo", 2023-06-27T12:38:02.437 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_default_realm_info_oid": "default.realm", 2023-06-27T12:38:02.437 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_default_region_info_oid": "default.region", 2023-06-27T12:38:02.437 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_default_zone_info_oid": "default.zone", 2023-06-27T12:38:02.437 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2023-06-27T12:38:02.438 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_defer_to_bucket_acls": "", 2023-06-27T12:38:02.438 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_delete_multi_obj_max_num": "1000", 2023-06-27T12:38:02.438 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_admin_lim": "0.000000", 2023-06-27T12:38:02.438 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_admin_res": "100.000000", 2023-06-27T12:38:02.438 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2023-06-27T12:38:02.438 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_auth_lim": "0.000000", 2023-06-27T12:38:02.438 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_auth_res": "200.000000", 2023-06-27T12:38:02.439 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2023-06-27T12:38:02.439 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_data_lim": "0.000000", 2023-06-27T12:38:02.439 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_data_res": "500.000000", 2023-06-27T12:38:02.439 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_data_wgt": "500.000000", 2023-06-27T12:38:02.439 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2023-06-27T12:38:02.439 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_metadata_res": "500.000000", 2023-06-27T12:38:02.439 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2023-06-27T12:38:02.439 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dns_name": "", 2023-06-27T12:38:02.440 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dns_s3website_name": "", 2023-06-27T12:38:02.440 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dynamic_resharding": "true", 2023-06-27T12:38:02.440 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2023-06-27T12:38:02.440 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_gc_threads": "true", 2023-06-27T12:38:02.440 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_lc_threads": "true", 2023-06-27T12:38:02.440 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_ops_log": "false", 2023-06-27T12:38:02.440 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_quota_threads": "true", 2023-06-27T12:38:02.440 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_static_website": "false", 2023-06-27T12:38:02.441 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_usage_log": "false", 2023-06-27T12:38:02.441 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enforce_swift_acls": "true", 2023-06-27T12:38:02.441 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_exit_timeout_secs": "120", 2023-06-27T12:38:02.441 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_expose_bucket": "false", 2023-06-27T12:38:02.441 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_extended_http_attrs": "", 2023-06-27T12:38:02.441 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_filter": "none", 2023-06-27T12:38:02.441 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2023-06-27T12:38:02.441 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_frontends": "beast port=7480", 2023-06-27T12:38:02.442 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_concurrent_io": "10", 2023-06-27T12:38:02.442 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_deferred": "50", 2023-06-27T12:38:02.442 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2023-06-27T12:38:02.442 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_objs": "32", 2023-06-27T12:38:02.442 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_queue_size": "134213632", 2023-06-27T12:38:02.442 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_trim_chunk": "16", 2023-06-27T12:38:02.442 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_obj_min_wait": "7200", 2023-06-27T12:38:02.442 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_processor_max_time": "3600", 2023-06-27T12:38:02.443 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_processor_period": "3600", 2023-06-27T12:38:02.443 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_get_obj_max_req_size": "4194304", 2023-06-27T12:38:02.443 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_get_obj_window_size": "16777216", 2023-06-27T12:38:02.443 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_healthcheck_disabling_path": "", 2023-06-27T12:38:02.443 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ignore_get_invalid_range": "false", 2023-06-27T12:38:02.443 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_init_timeout": "300", 2023-06-27T12:38:02.443 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2023-06-27T12:38:02.443 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2023-06-27T12:38:02.444 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_accepted_admin_roles": "", 2023-06-27T12:38:02.444 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_accepted_reader_roles": "", 2023-06-27T12:38:02.444 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2023-06-27T12:38:02.444 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_domain": "", 2023-06-27T12:38:02.444 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_password": "", 2023-06-27T12:38:02.444 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_password_path": "", 2023-06-27T12:38:02.444 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_project": "", 2023-06-27T12:38:02.445 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_tenant": "", 2023-06-27T12:38:02.445 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_token": "", 2023-06-27T12:38:02.445 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_token_path": "", 2023-06-27T12:38:02.445 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_user": "", 2023-06-27T12:38:02.445 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_api_version": "2", 2023-06-27T12:38:02.445 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_domain": "", 2023-06-27T12:38:02.445 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_password": "", 2023-06-27T12:38:02.445 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_project": "", 2023-06-27T12:38:02.446 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_tenant": "", 2023-06-27T12:38:02.446 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_user": "", 2023-06-27T12:38:02.446 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2023-06-27T12:38:02.446 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_implicit_tenants": "false", 2023-06-27T12:38:02.446 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2023-06-27T12:38:02.446 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_service_token_enabled": "false", 2023-06-27T12:38:02.446 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_token_cache_size": "10000", 2023-06-27T12:38:02.446 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_url": "", 2023-06-27T12:38:02.447 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_verify_ssl": "true", 2023-06-27T12:38:02.447 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_debug_interval": "-1", 2023-06-27T12:38:02.447 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_lock_max_time": "90", 2023-06-27T12:38:02.447 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_max_objs": "32", 2023-06-27T12:38:02.447 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_max_rules": "1000", 2023-06-27T12:38:02.447 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_max_worker": "3", 2023-06-27T12:38:02.447 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_max_wp_worker": "3", 2023-06-27T12:38:02.447 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_thread_delay": "0", 2023-06-27T12:38:02.448 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2023-06-27T12:38:02.448 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_dnattr": "uid", 2023-06-27T12:38:02.448 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2023-06-27T12:38:02.448 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_searchfilter": "", 2023-06-27T12:38:02.448 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2023-06-27T12:38:02.448 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_uri": "ldaps://", 2023-06-27T12:38:02.448 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2023-06-27T12:38:02.448 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_list_bucket_min_readahead": "1000", 2023-06-27T12:38:02.449 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_list_buckets_max_chunk": "1000", 2023-06-27T12:38:02.449 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_log_http_headers": "", 2023-06-27T12:38:02.449 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_log_nonexistent_bucket": "false", 2023-06-27T12:38:02.449 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2023-06-27T12:38:02.449 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_log_object_name_utc": "false", 2023-06-27T12:38:02.449 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2023-06-27T12:38:02.449 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_attr_name_len": "0", 2023-06-27T12:38:02.449 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_attr_size": "0", 2023-06-27T12:38:02.450 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_attrs_num_in_req": "0", 2023-06-27T12:38:02.450 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_chunk_size": "4194304", 2023-06-27T12:38:02.450 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_concurrent_requests": "1024", 2023-06-27T12:38:02.450 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2023-06-27T12:38:02.450 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_dynamic_shards": "1999", 2023-06-27T12:38:02.450 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_listing_results": "1000", 2023-06-27T12:38:02.450 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_notify_retries": "3", 2023-06-27T12:38:02.450 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_objs_per_shard": "100000", 2023-06-27T12:38:02.451 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_put_param_size": "1048576", 2023-06-27T12:38:02.451 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_put_size": "5368709120", 2023-06-27T12:38:02.451 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_slo_entries": "1000", 2023-06-27T12:38:02.451 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_md_log_max_shards": "64", 2023-06-27T12:38:02.451 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_md_notify_interval_msec": "200", 2023-06-27T12:38:02.451 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_meta_sync_poll_interval": "20", 2023-06-27T12:38:02.451 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_meta_sync_spawn_window": "20", 2023-06-27T12:38:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_mime_types_file": "/etc/mime.types", 2023-06-27T12:38:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_mp_lock_max_time": "600", 2023-06-27T12:38:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_multi_obj_del_max_aio": "16", 2023-06-27T12:38:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_multipart_min_part_size": "5242880", 2023-06-27T12:38:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_multipart_part_upload_limit": "10000", 2023-06-27T12:38:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_fhcache_partitions": "3", 2023-06-27T12:38:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_fhcache_size": "2017", 2023-06-27T12:38:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_frontends": "rgw-nfs", 2023-06-27T12:38:02.453 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2023-06-27T12:38:02.453 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_lru_lanes": "5", 2023-06-27T12:38:02.453 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_max_gc": "300", 2023-06-27T12:38:02.453 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_namespace_expire_secs": "300", 2023-06-27T12:38:02.453 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_run_gc_threads": "false", 2023-06-27T12:38:02.453 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_run_lc_threads": "false", 2023-06-27T12:38:02.453 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_run_quota_threads": "false", 2023-06-27T12:38:02.453 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_run_sync_thread": "false", 2023-06-27T12:38:02.454 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_s3_fast_attrs": "false", 2023-06-27T12:38:02.454 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_write_completion_interval_s": "10", 2023-06-27T12:38:02.454 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_num_async_rados_threads": "32", 2023-06-27T12:38:02.454 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_num_control_oids": "8", 2023-06-27T12:38:02.454 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_numa_node": "-1", 2023-06-27T12:38:02.454 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_obj_stripe_size": "4194304", 2023-06-27T12:38:02.454 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_obj_tombstone_cache_size": "1000", 2023-06-27T12:38:02.455 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_objexp_chunk_size": "100", 2023-06-27T12:38:02.455 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_objexp_gc_interval": "600", 2023-06-27T12:38:02.455 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_objexp_hints_num_shards": "127", 2023-06-27T12:38:02.455 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_olh_pending_timeout_sec": "3600", 2023-06-27T12:38:02.455 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_op_thread_suicide_timeout": "0", 2023-06-27T12:38:02.455 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_op_thread_timeout": "600", 2023-06-27T12:38:02.455 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_op_tracing": "false", 2023-06-27T12:38:02.455 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_opa_token": "", 2023-06-27T12:38:02.456 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_opa_url": "", 2023-06-27T12:38:02.456 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_opa_verify_ssl": "true", 2023-06-27T12:38:02.456 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ops_log_data_backlog": "5242880", 2023-06-27T12:38:02.456 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2023-06-27T12:38:02.456 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ops_log_rados": "false", 2023-06-27T12:38:02.456 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ops_log_socket_path": "", 2023-06-27T12:38:02.456 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_override_bucket_index_max_shards": "0", 2023-06-27T12:38:02.456 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2023-06-27T12:38:02.457 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2023-06-27T12:38:02.457 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_period_push_interval": "2.000000", 2023-06-27T12:38:02.457 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_period_push_interval_max": "30.000000", 2023-06-27T12:38:02.457 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_period_root_pool": ".rgw.root", 2023-06-27T12:38:02.457 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_policy_reject_invalid_principals": "true", 2023-06-27T12:38:02.457 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_print_continue": "true", 2023-06-27T12:38:02.457 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_print_prohibited_content_length": "false", 2023-06-27T12:38:02.457 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_put_obj_max_window_size": "67108864", 2023-06-27T12:38:02.458 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_put_obj_min_window_size": "16777216", 2023-06-27T12:38:02.458 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2023-06-27T12:38:02.458 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_rados_pool_recovery_priority": "5", 2023-06-27T12:38:02.458 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_rados_tracing": "false", 2023-06-27T12:38:02.458 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_realm": "", 2023-06-27T12:38:02.458 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_realm_id": "", 2023-06-27T12:38:02.458 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_realm_root_pool": ".rgw.root", 2023-06-27T12:38:02.458 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_region": "", 2023-06-27T12:38:02.459 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_region_root_pool": ".rgw.root", 2023-06-27T12:38:02.459 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_relaxed_region_enforcement": "false", 2023-06-27T12:38:02.459 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_relaxed_s3_bucket_names": "false", 2023-06-27T12:38:02.459 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2023-06-27T12:38:02.459 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_request_uri": "", 2023-06-27T12:38:02.459 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_batch_size": "64", 2023-06-27T12:38:02.459 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_bucket_lock_duration": "360", 2023-06-27T12:38:02.459 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_max_aio": "128", 2023-06-27T12:38:02.460 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_num_logs": "16", 2023-06-27T12:38:02.460 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_thread_interval": "600", 2023-06-27T12:38:02.460 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_resolve_cname": "false", 2023-06-27T12:38:02.460 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_rest_getusage_op_compat": "false", 2023-06-27T12:38:02.460 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_run_sync_thread": "true", 2023-06-27T12:38:02.460 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_order": "sts, external, local", 2023-06-27T12:38:02.460 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_use_keystone": "false", 2023-06-27T12:38:02.461 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_use_ldap": "false", 2023-06-27T12:38:02.461 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_use_rados": "true", 2023-06-27T12:38:02.461 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_use_sts": "false", 2023-06-27T12:38:02.461 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_client_max_sig_ver": "-1", 2023-06-27T12:38:02.461 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_success_create_obj_status": "0", 2023-06-27T12:38:02.461 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_safe_max_objects_per_shard": "102400", 2023-06-27T12:38:02.461 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_scheduler_type": "throttler", 2023-06-27T12:38:02.461 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_script_uri": "", 2023-06-27T12:38:02.462 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_service_provider_name": "", 2023-06-27T12:38:02.462 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_shard_warning_threshold": "90.000000", 2023-06-27T12:38:02.462 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_client_id": "", 2023-06-27T12:38:02.462 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_client_secret": "", 2023-06-27T12:38:02.462 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_entry": "sts", 2023-06-27T12:38:02.462 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_key": "sts", 2023-06-27T12:38:02.462 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_max_session_duration": "43200", 2023-06-27T12:38:02.462 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_min_session_duration": "900", 2023-06-27T12:38:02.463 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_token_introspection_url": "", 2023-06-27T12:38:02.463 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_account_in_url": "false", 2023-06-27T12:38:02.463 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_auth_entry": "auth", 2023-06-27T12:38:02.463 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_auth_url": "", 2023-06-27T12:38:02.463 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_custom_header": "", 2023-06-27T12:38:02.463 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_enforce_content_length": "false", 2023-06-27T12:38:02.463 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_need_stats": "true", 2023-06-27T12:38:02.463 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_tenant_name": "", 2023-06-27T12:38:02.464 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_token_expiration": "86400", 2023-06-27T12:38:02.464 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_url": "", 2023-06-27T12:38:02.464 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_url_prefix": "swift", 2023-06-27T12:38:02.464 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_versioning_enabled": "false", 2023-06-27T12:38:02.464 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2023-06-27T12:38:02.464 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2023-06-27T12:38:02.464 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_lease_period": "120", 2023-06-27T12:38:02.465 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2023-06-27T12:38:02.465 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_log_trim_interval": "1200", 2023-06-27T12:38:02.465 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_log_trim_max_buckets": "16", 2023-06-27T12:38:02.465 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2023-06-27T12:38:02.465 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2023-06-27T12:38:02.465 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_obj_etag_verify": "false", 2023-06-27T12:38:02.465 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_trace_history_size": "4096", 2023-06-27T12:38:02.465 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_trace_per_node_log_size": "32", 2023-06-27T12:38:02.466 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2023-06-27T12:38:02.466 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_thread_pool_size": "512", 2023-06-27T12:38:02.466 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_comment": "", 2023-06-27T12:38:02.466 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_createby": "", 2023-06-27T12:38:02.466 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_encoding": "", 2023-06-27T12:38:02.466 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_flag": "false", 2023-06-27T12:38:02.466 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_max_size": "5368709120", 2023-06-27T12:38:02.466 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_origin": "", 2023-06-27T12:38:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_sha_unit": "524288", 2023-06-27T12:38:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_tracker": "", 2023-06-27T12:38:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_trust_forwarded_https": "false", 2023-06-27T12:38:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_usage_log_flush_threshold": "1024", 2023-06-27T12:38:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_usage_log_tick_interval": "30", 2023-06-27T12:38:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_usage_max_shards": "32", 2023-06-27T12:38:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_usage_max_user_shards": "1", 2023-06-27T12:38:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_use_opa_authz": "false", 2023-06-27T12:38:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_default_quota_max_objects": "-1", 2023-06-27T12:38:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_default_quota_max_size": "-1", 2023-06-27T12:38:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_max_buckets": "1000", 2023-06-27T12:38:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_policies_max_num": "100", 2023-06-27T12:38:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2023-06-27T12:38:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_quota_sync_idle_users": "false", 2023-06-27T12:38:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_quota_sync_interval": "86400", 2023-06-27T12:38:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_quota_sync_wait_time": "86400", 2023-06-27T12:38:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_unique_email": "true", 2023-06-27T12:38:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_verify_ssl": "true", 2023-06-27T12:38:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_website_routing_rules_max_num": "50", 2023-06-27T12:38:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zone": "", 2023-06-27T12:38:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zone_id": "", 2023-06-27T12:38:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zone_root_pool": ".rgw.root", 2023-06-27T12:38:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zonegroup": "", 2023-06-27T12:38:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zonegroup_id": "", 2023-06-27T12:38:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2023-06-27T12:38:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "rgwlc_auto_session_clear": "true", 2023-06-27T12:38:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "rgwlc_skip_bucket_step": "false", 2023-06-27T12:38:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_block_size": "4096", 2023-06-27T12:38:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_bloom_bits_per_key": "20", 2023-06-27T12:38:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2023-06-27T12:38:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2023-06-27T12:38:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_row_ratio": "0.000000", 2023-06-27T12:38:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_shard_bits": "4", 2023-06-27T12:38:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_size": "536870912", 2023-06-27T12:38:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_type": "binned_lru", 2023-06-27T12:38:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cf_compact_on_deletion": "true", 2023-06-27T12:38:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2023-06-27T12:38:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2023-06-27T12:38:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_collect_compaction_stats": "false", 2023-06-27T12:38:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_collect_extended_stats": "false", 2023-06-27T12:38:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_collect_memory_stats": "false", 2023-06-27T12:38:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_delete_range_threshold": "1048576", 2023-06-27T12:38:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_index_type": "binary_search", 2023-06-27T12:38:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_log_to_ceph_log": "true", 2023-06-27T12:38:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_metadata_block_size": "4096", 2023-06-27T12:38:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_partition_filters": "false", 2023-06-27T12:38:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_perf": "false", 2023-06-27T12:38:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2023-06-27T12:38:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "rotating_keys_bootstrap_timeout": "30", 2023-06-27T12:38:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "rotating_keys_renewal_timeout": "10", 2023-06-27T12:38:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "run_dir": "/var/run/ceph", 2023-06-27T12:38:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_block_create": "true", 2023-06-27T12:38:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_cache_lru_size": "67108864", 2023-06-27T12:38:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_cbjournal_size": "5368709120", 2023-06-27T12:38:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_default_max_object_size": "16777216", 2023-06-27T12:38:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_default_object_metadata_reservation": "16777216", 2023-06-27T12:38:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_device_size": "53687091200", 2023-06-27T12:38:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_journal_batch_capacity": "16", 2023-06-27T12:38:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_journal_batch_flush_size": "16777216", 2023-06-27T12:38:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2023-06-27T12:38:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_journal_iodepth_limit": "5", 2023-06-27T12:38:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_main_device_type": "SSD", 2023-06-27T12:38:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_max_concurrent_transactions": "8", 2023-06-27T12:38:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2023-06-27T12:38:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2023-06-27T12:38:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2023-06-27T12:38:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_obj_data_write_amplification": "1.250000", 2023-06-27T12:38:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "seastore_segment_size": "67108864", 2023-06-27T12:38:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "set_keepcaps": "false", 2023-06-27T12:38:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "setgroup": "", 2023-06-27T12:38:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "setuser": "", 2023-06-27T12:38:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "setuser_match_path": "", 2023-06-27T12:38:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_misplaced_ratio": "0.050000", 2023-06-27T12:38:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "thp": "false", 2023-06-27T12:38:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "threadpool_default_timeout": "60", 2023-06-27T12:38:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "threadpool_empty_queue_max_wait": "2", 2023-06-27T12:38:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "throttler_perf_counter": "true" 2023-06-27T12:38:02.476 INFO:teuthology.orchestra.run.smithi138.stdout:} 2023-06-27T12:38:02.478 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2023-06-27T12:38:02.603 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:02.598+0000 7f9a181fd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:02.606 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:02.602+0000 7f9a181fd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:02.619 INFO:teuthology.orchestra.run.smithi138.stdout:{} 2023-06-27T12:38:02.620 INFO:teuthology.orchestra.run.smithi138.stderr:mon_osd_down_out_interval = '0' (not observed, change may require restart) 2023-06-27T12:38:02.631 DEBUG:teuthology.orchestra.run.smithi138:> type ceph-objectstore-tool 2023-06-27T12:38:02.647 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2023-06-27T12:38:02.648 DEBUG:teuthology.orchestra.run.smithi159:> type ceph-objectstore-tool 2023-06-27T12:38:02.669 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2023-06-27T12:38:02.669 INFO:teuthology.run_tasks:Running task rados... 2023-06-27T12:38:02.685 INFO:tasks.rados:Beginning rados... 2023-06-27T12:38:02.685 DEBUG:teuthology.run_tasks:Unwinding manager rados 2023-06-27T12:38:02.693 INFO:tasks.rados:joining rados 2023-06-27T12:38:02.694 INFO:tasks.thrashosds.thrasher:starting do_thrash 2023-06-27T12:38:02.694 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:02.694 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:02.694 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:02.695 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2023-06-27T12:38:02.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:02.695 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2023-06-27T12:38:02.696 INFO:tasks.rados:starting run 0 out of 1 2023-06-27T12:38:02.696 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2023-06-27T12:38:02.696 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2023-06-27T12:38:02.831 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:02.824+0000 7f6c7b3c5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:02.835 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:02.831+0000 7f6c7b3c5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:02.837 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:02.832+0000 7feee0373700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:02.844 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:02.832+0000 7f816846c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:02.845 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:02.841+0000 7feee0373700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:02.848 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:02.845+0000 7f816846c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:03.063 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2023-06-27T12:38:01.905708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107736,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379624,"statfs":{"total":386547056640,"available":386436734976,"internally_reserved":0,"allocated":417792,"data_stored":1854788,"data_compressed":82433,"data_compressed_allocated":110592,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":9,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.907306"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.916103+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:37:57.916103+0000","last_peered":"2023-06-27T12:37:57.916103+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:37:57.916103+0000","last_undegraded":"2023-06-27T12:37:57.916103+0000","last_fullsized":"2023-06-27T12:37:57.916103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.830069+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:37:58.830069+0000","last_peered":"2023-06-27T12:37:58.830069+0000","last_clean":"2023-06-27T12:37:58.830069+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:37:58.830069+0000","last_undegraded":"2023-06-27T12:37:58.830069+0000","last_fullsized":"2023-06-27T12:37:58.830069+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.934379+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:37:57.934379+0000","last_peered":"2023-06-27T12:37:57.934379+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:37:57.934379+0000","last_undegraded":"2023-06-27T12:37:57.934379+0000","last_fullsized":"2023-06-27T12:37:57.934379+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.952219+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:37:57.952219+0000","last_peered":"2023-06-27T12:37:57.952219+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:37:57.952219+0000","last_undegraded":"2023-06-27T12:37:57.952219+0000","last_fullsized":"2023-06-27T12:37:57.952219+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"14'92","reported_seq":136,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:59.140819+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:37:59.140819+0000","last_peered":"2023-06-27T12:37:59.140819+0000","last_clean":"2023-06-27T12:37:59.140819+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:37:59.140819+0000","last_undegraded":"2023-06-27T12:37:59.140819+0000","last_fullsized":"2023-06-27T12:37:59.140819+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.911079+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:37:57.911079+0000","last_peered":"2023-06-27T12:37:57.911079+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:37:57.911079+0000","last_undegraded":"2023-06-27T12:37:57.911079+0000","last_fullsized":"2023-06-27T12:37:57.911079+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.881722+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:37:57.881722+0000","last_peered":"2023-06-27T12:37:57.881722+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:37:57.881722+0000","last_undegraded":"2023-06-27T12:37:57.881722+0000","last_fullsized":"2023-06-27T12:37:57.881722+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-06-27T12:37:57.941686+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:37:57.941686+0000","last_peered":"2023-06-27T12:37:57.941686+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:37:57.941686+0000","last_undegraded":"2023-06-27T12:37:57.941686+0000","last_fullsized":"2023-06-27T12:37:57.941686+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-06-27T12:37:58.825454+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:37:58.825454+0000","last_peered":"2023-06-27T12:37:58.825454+0000","last_clean":"2023-06-27T12:37:58.825454+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:37:58.825454+0000","last_undegraded":"2023-06-27T12:37:58.825454+0000","last_fullsized":"2023-06-27T12:37:58.825454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":89,"num_write":132,"num_write_kb":1765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27478,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26948,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344892,"statfs":{"total":96636764160,"available":96609169408,"internally_reserved":0,"allocated":118784,"data_stored":611289,"data_compressed":27477,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26884,"kb_used_data":60,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344956,"statfs":{"total":96636764160,"available":96609234944,"internally_reserved":0,"allocated":61440,"data_stored":20921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:03.375 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'unique_pool_0' created 2023-06-27T12:38:03.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2023-06-27T12:38:03.529 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:03.517+0000 7f0a92e5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:03.532 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:03.528+0000 7f0a92e5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:04.386 INFO:teuthology.orchestra.run.smithi138.stderr:enabled application 'rados' on pool 'unique_pool_0' 2023-06-27T12:38:04.926 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:05.052 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:05.047+0000 7fa75b949700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:05.055 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:05.051+0000 7fa75b949700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:05.400 DEBUG:teuthology.orchestra.run.smithi138:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --localize-reads --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 0 --op copy_from 0 --op write_excl 50 --pool unique_pool_0 2023-06-27T12:38:05.437 INFO:tasks.rados.rados.0.smithi138.stdout:adding op weight read -> 100 2023-06-27T12:38:05.437 INFO:tasks.rados.rados.0.smithi138.stdout:adding op weight write -> 50 2023-06-27T12:38:05.437 INFO:tasks.rados.rados.0.smithi138.stdout:adding op weight delete -> 50 2023-06-27T12:38:05.437 INFO:tasks.rados.rados.0.smithi138.stdout:adding op weight snap_create -> 50 2023-06-27T12:38:05.438 INFO:tasks.rados.rados.0.smithi138.stdout:adding op weight snap_remove -> 50 2023-06-27T12:38:05.438 INFO:tasks.rados.rados.0.smithi138.stdout:adding op weight write_excl -> 50 2023-06-27T12:38:05.438 INFO:tasks.rados.rados.0.smithi138.stdout:ceph version 18.0.0-4614-gba3fff72 (ba3fff7216d65a7042e2ef64fe924764dc57a097) reef (dev) 2023-06-27T12:38:05.438 INFO:tasks.rados.rados.0.smithi138.stdout:Configuration: 2023-06-27T12:38:05.438 INFO:tasks.rados.rados.0.smithi138.stdout: Number of operations: 4000 2023-06-27T12:38:05.438 INFO:tasks.rados.rados.0.smithi138.stdout: Number of objects: 50 2023-06-27T12:38:05.439 INFO:tasks.rados.rados.0.smithi138.stdout: Max in flight operations: 16 2023-06-27T12:38:05.439 INFO:tasks.rados.rados.0.smithi138.stdout: Object size (in bytes): 4000000 2023-06-27T12:38:05.439 INFO:tasks.rados.rados.0.smithi138.stdout: Write stride min: 400000 2023-06-27T12:38:05.439 INFO:tasks.rados.rados.0.smithi138.stdout: Write stride max: 800000 2023-06-27T12:38:06.389 INFO:tasks.rados.rados.0.smithi138.stdout:1: write initial oid 1 2023-06-27T12:38:06.389 INFO:tasks.rados.rados.0.smithi138.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2023-06-27T12:38:06.394 INFO:tasks.rados.rados.0.smithi138.stdout:1: writing smithi138138488-1 from 416807 to 892056 tid 1 2023-06-27T12:38:06.400 INFO:tasks.rados.rados.0.smithi138.stdout:1: writing smithi138138488-1 from 1542129 to 2016807 tid 2 2023-06-27T12:38:06.400 INFO:tasks.rados.rados.0.smithi138.stdout:2: write initial oid 2 2023-06-27T12:38:06.401 INFO:tasks.rados.rados.0.smithi138.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2023-06-27T12:38:06.406 INFO:tasks.rados.rados.0.smithi138.stdout:2: writing smithi138138488-2 from 416807 to 892056 tid 1 2023-06-27T12:38:06.414 INFO:tasks.rados.rados.0.smithi138.stdout:2: writing smithi138138488-2 from 1542129 to 2016807 tid 2 2023-06-27T12:38:06.415 INFO:tasks.rados.rados.0.smithi138.stdout:3: write initial oid 3 2023-06-27T12:38:06.415 INFO:tasks.rados.rados.0.smithi138.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2023-06-27T12:38:06.420 INFO:tasks.rados.rados.0.smithi138.stdout:3: writing smithi138138488-3 from 433614 to 984112 tid 1 2023-06-27T12:38:06.427 INFO:tasks.rados.rados.0.smithi138.stdout:3: writing smithi138138488-3 from 1600611 to 2033614 tid 2 2023-06-27T12:38:06.428 INFO:tasks.rados.rados.0.smithi138.stdout:4: write initial oid 4 2023-06-27T12:38:06.428 INFO:tasks.rados.rados.0.smithi138.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2023-06-27T12:38:06.435 INFO:tasks.rados.rados.0.smithi138.stdout:4: writing smithi138138488-4 from 450421 to 1076168 tid 1 2023-06-27T12:38:06.441 INFO:tasks.rados.rados.0.smithi138.stdout:4: writing smithi138138488-4 from 1659093 to 2050421 tid 2 2023-06-27T12:38:06.441 INFO:tasks.rados.rados.0.smithi138.stdout:5: write initial oid 5 2023-06-27T12:38:06.441 INFO:tasks.rados.rados.0.smithi138.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2023-06-27T12:38:06.448 INFO:tasks.rados.rados.0.smithi138.stdout:5: writing smithi138138488-5 from 467228 to 1168224 tid 1 2023-06-27T12:38:06.453 INFO:tasks.rados.rados.0.smithi138.stdout:5: writing smithi138138488-5 from 1717575 to 2067228 tid 2 2023-06-27T12:38:06.453 INFO:tasks.rados.rados.0.smithi138.stdout:6: write initial oid 6 2023-06-27T12:38:06.453 INFO:tasks.rados.rados.0.smithi138.stdout:1: finishing write tid 1 to smithi138138488-1 2023-06-27T12:38:06.453 INFO:tasks.rados.rados.0.smithi138.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2023-06-27T12:38:06.460 INFO:tasks.rados.rados.0.smithi138.stdout:6: writing smithi138138488-6 from 484035 to 1260280 tid 1 2023-06-27T12:38:06.465 INFO:tasks.rados.rados.0.smithi138.stdout:6: writing smithi138138488-6 from 2059704 to 2084035 tid 2 2023-06-27T12:38:06.465 INFO:tasks.rados.rados.0.smithi138.stdout:7: write initial oid 7 2023-06-27T12:38:06.466 INFO:tasks.rados.rados.0.smithi138.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2023-06-27T12:38:06.471 INFO:tasks.rados.rados.0.smithi138.stdout:7: writing smithi138138488-7 from 500842 to 952336 tid 1 2023-06-27T12:38:06.477 INFO:tasks.rados.rados.0.smithi138.stdout:7: writing smithi138138488-7 from 1718186 to 2100842 tid 2 2023-06-27T12:38:06.478 INFO:tasks.rados.rados.0.smithi138.stdout:8: write initial oid 8 2023-06-27T12:38:06.478 INFO:tasks.rados.rados.0.smithi138.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2023-06-27T12:38:06.484 INFO:tasks.rados.rados.0.smithi138.stdout:8: writing smithi138138488-8 from 517649 to 1044392 tid 1 2023-06-27T12:38:06.490 INFO:tasks.rados.rados.0.smithi138.stdout:8: writing smithi138138488-8 from 1776668 to 2117649 tid 2 2023-06-27T12:38:06.490 INFO:tasks.rados.rados.0.smithi138.stdout:9: write initial oid 9 2023-06-27T12:38:06.490 INFO:tasks.rados.rados.0.smithi138.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2023-06-27T12:38:06.498 INFO:tasks.rados.rados.0.smithi138.stdout:9: writing smithi138138488-9 from 534456 to 1252801 tid 1 2023-06-27T12:38:06.503 INFO:tasks.rados.rados.0.smithi138.stdout:9: writing smithi138138488-9 from 1951503 to 2134456 tid 2 2023-06-27T12:38:06.503 INFO:tasks.rados.rados.0.smithi138.stdout:10: write initial oid 10 2023-06-27T12:38:06.503 INFO:tasks.rados.rados.0.smithi138.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2023-06-27T12:38:06.510 INFO:tasks.rados.rados.0.smithi138.stdout:10: writing smithi138138488-10 from 551263 to 1344857 tid 1 2023-06-27T12:38:06.515 INFO:tasks.rados.rados.0.smithi138.stdout:10: writing smithi138138488-10 from 1893632 to 2151263 tid 2 2023-06-27T12:38:06.515 INFO:tasks.rados.rados.0.smithi138.stdout:11: write initial oid 11 2023-06-27T12:38:06.516 INFO:tasks.rados.rados.0.smithi138.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2023-06-27T12:38:06.521 INFO:tasks.rados.rados.0.smithi138.stdout:11: writing smithi138138488-11 from 568070 to 1036913 tid 1 2023-06-27T12:38:06.529 INFO:tasks.rados.rados.0.smithi138.stdout:11: writing smithi138138488-11 from 1552114 to 2168070 tid 2 2023-06-27T12:38:06.529 INFO:tasks.rados.rados.0.smithi138.stdout:12: write initial oid 12 2023-06-27T12:38:06.530 INFO:tasks.rados.rados.0.smithi138.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2023-06-27T12:38:06.535 INFO:tasks.rados.rados.0.smithi138.stdout:12: writing smithi138138488-12 from 584877 to 1128969 tid 1 2023-06-27T12:38:06.541 INFO:tasks.rados.rados.0.smithi138.stdout:12: writing smithi138138488-12 from 1610596 to 2172599 tid 2 2023-06-27T12:38:06.542 INFO:tasks.rados.rados.0.smithi138.stdout:13: write initial oid 13 2023-06-27T12:38:06.542 INFO:tasks.rados.rados.0.smithi138.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2023-06-27T12:38:06.548 INFO:tasks.rados.rados.0.smithi138.stdout:13: writing smithi138138488-13 from 601684 to 1221025 tid 1 2023-06-27T12:38:06.554 INFO:tasks.rados.rados.0.smithi138.stdout:13: writing smithi138138488-13 from 1669078 to 2201684 tid 2 2023-06-27T12:38:06.554 INFO:tasks.rados.rados.0.smithi138.stdout:14: write initial oid 14 2023-06-27T12:38:06.554 INFO:tasks.rados.rados.0.smithi138.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2023-06-27T12:38:06.561 INFO:tasks.rados.rados.0.smithi138.stdout:14: writing smithi138138488-14 from 618491 to 1313081 tid 1 2023-06-27T12:38:06.567 INFO:tasks.rados.rados.0.smithi138.stdout:14: writing smithi138138488-14 from 2011207 to 2218491 tid 2 2023-06-27T12:38:06.567 INFO:tasks.rados.rados.0.smithi138.stdout:15: write initial oid 15 2023-06-27T12:38:06.568 INFO:tasks.rados.rados.0.smithi138.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2023-06-27T12:38:06.575 INFO:tasks.rados.rados.0.smithi138.stdout:15: writing smithi138138488-15 from 635298 to 1405137 tid 1 2023-06-27T12:38:06.579 INFO:tasks.rados.rados.0.smithi138.stdout:15: writing smithi138138488-15 from 2069689 to 2235298 tid 2 2023-06-27T12:38:06.580 INFO:tasks.rados.rados.0.smithi138.stdout:16: write initial oid 16 2023-06-27T12:38:06.580 INFO:tasks.rados.rados.0.smithi138.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2023-06-27T12:38:06.586 INFO:tasks.rados.rados.0.smithi138.stdout:16: writing smithi138138488-16 from 652105 to 1097193 tid 1 2023-06-27T12:38:06.593 INFO:tasks.rados.rados.0.smithi138.stdout:16: writing smithi138138488-16 from 1728171 to 2181159 tid 2 2023-06-27T12:38:06.594 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.594 INFO:tasks.rados.rados.0.smithi138.stdout:2: finishing write tid 1 to smithi138138488-2 2023-06-27T12:38:06.594 INFO:tasks.rados.rados.0.smithi138.stdout:1: finishing write tid 2 to smithi138138488-1 2023-06-27T12:38:06.595 INFO:tasks.rados.rados.0.smithi138.stdout:1: finishing write tid 3 to smithi138138488-1 2023-06-27T12:38:06.595 INFO:tasks.rados.rados.0.smithi138.stdout:1: finishing write tid 4 to smithi138138488-1 2023-06-27T12:38:06.595 INFO:tasks.rados.rados.0.smithi138.stdout:1: oid 1 updating version 0 to 3 2023-06-27T12:38:06.595 INFO:tasks.rados.rados.0.smithi138.stdout:1: oid 1 version 3 is already newer than 1 2023-06-27T12:38:06.596 INFO:tasks.rados.rados.0.smithi138.stdout:1: oid 1 version 3 is already newer than 3 2023-06-27T12:38:06.596 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2023-06-27T12:38:06.596 INFO:tasks.rados.rados.0.smithi138.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2023-06-27T12:38:06.596 INFO:tasks.rados.rados.0.smithi138.stdout:2: finishing write tid 2 to smithi138138488-2 2023-06-27T12:38:06.597 INFO:tasks.rados.rados.0.smithi138.stdout:1: done (15 left) 2023-06-27T12:38:06.597 INFO:tasks.rados.rados.0.smithi138.stdout:17: write initial oid 17 2023-06-27T12:38:06.597 INFO:tasks.rados.rados.0.smithi138.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2023-06-27T12:38:06.600 INFO:tasks.rados.rados.0.smithi138.stdout:17: writing smithi138138488-17 from 668912 to 1305602 tid 1 2023-06-27T12:38:06.607 INFO:tasks.rados.rados.0.smithi138.stdout:17: writing smithi138138488-17 from 1903006 to 2268912 tid 2 2023-06-27T12:38:06.607 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.607 INFO:tasks.rados.rados.0.smithi138.stdout:3: finishing write tid 1 to smithi138138488-3 2023-06-27T12:38:06.607 INFO:tasks.rados.rados.0.smithi138.stdout:2: finishing write tid 3 to smithi138138488-2 2023-06-27T12:38:06.608 INFO:tasks.rados.rados.0.smithi138.stdout:6: finishing write tid 1 to smithi138138488-6 2023-06-27T12:38:06.608 INFO:tasks.rados.rados.0.smithi138.stdout:2: finishing write tid 4 to smithi138138488-2 2023-06-27T12:38:06.608 INFO:tasks.rados.rados.0.smithi138.stdout:2: oid 2 updating version 0 to 3 2023-06-27T12:38:06.608 INFO:tasks.rados.rados.0.smithi138.stdout:2: oid 2 version 3 is already newer than 1 2023-06-27T12:38:06.608 INFO:tasks.rados.rados.0.smithi138.stdout:2: oid 2 version 3 is already newer than 3 2023-06-27T12:38:06.608 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2023-06-27T12:38:06.609 INFO:tasks.rados.rados.0.smithi138.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2023-06-27T12:38:06.609 INFO:tasks.rados.rados.0.smithi138.stdout:3: finishing write tid 2 to smithi138138488-3 2023-06-27T12:38:06.609 INFO:tasks.rados.rados.0.smithi138.stdout:2: done (15 left) 2023-06-27T12:38:06.609 INFO:tasks.rados.rados.0.smithi138.stdout:18: write initial oid 18 2023-06-27T12:38:06.609 INFO:tasks.rados.rados.0.smithi138.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2023-06-27T12:38:06.614 INFO:tasks.rados.rados.0.smithi138.stdout:18: writing smithi138138488-18 from 685719 to 1397658 tid 1 2023-06-27T12:38:06.620 INFO:tasks.rados.rados.0.smithi138.stdout:18: writing smithi138138488-18 from 1845135 to 2285719 tid 2 2023-06-27T12:38:06.621 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.621 INFO:tasks.rados.rados.0.smithi138.stdout:4: finishing write tid 1 to smithi138138488-4 2023-06-27T12:38:06.621 INFO:tasks.rados.rados.0.smithi138.stdout:7: finishing write tid 1 to smithi138138488-7 2023-06-27T12:38:06.621 INFO:tasks.rados.rados.0.smithi138.stdout:3: finishing write tid 3 to smithi138138488-3 2023-06-27T12:38:06.621 INFO:tasks.rados.rados.0.smithi138.stdout:3: finishing write tid 4 to smithi138138488-3 2023-06-27T12:38:06.621 INFO:tasks.rados.rados.0.smithi138.stdout:3: oid 3 updating version 0 to 3 2023-06-27T12:38:06.621 INFO:tasks.rados.rados.0.smithi138.stdout:3: oid 3 version 3 is already newer than 3 2023-06-27T12:38:06.622 INFO:tasks.rados.rados.0.smithi138.stdout:3: oid 3 version 3 is already newer than 1 2023-06-27T12:38:06.622 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2023-06-27T12:38:06.622 INFO:tasks.rados.rados.0.smithi138.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2023-06-27T12:38:06.622 INFO:tasks.rados.rados.0.smithi138.stdout:9: finishing write tid 1 to smithi138138488-9 2023-06-27T12:38:06.622 INFO:tasks.rados.rados.0.smithi138.stdout:3: done (15 left) 2023-06-27T12:38:06.622 INFO:tasks.rados.rados.0.smithi138.stdout:19: write initial oid 19 2023-06-27T12:38:06.622 INFO:tasks.rados.rados.0.smithi138.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2023-06-27T12:38:06.629 INFO:tasks.rados.rados.0.smithi138.stdout:19: writing smithi138138488-19 from 702526 to 1489714 tid 1 2023-06-27T12:38:06.633 INFO:tasks.rados.rados.0.smithi138.stdout:19: writing smithi138138488-19 from 1903617 to 2302526 tid 2 2023-06-27T12:38:06.633 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.634 INFO:tasks.rados.rados.0.smithi138.stdout:11: finishing write tid 1 to smithi138138488-11 2023-06-27T12:38:06.634 INFO:tasks.rados.rados.0.smithi138.stdout:6: finishing write tid 2 to smithi138138488-6 2023-06-27T12:38:06.634 INFO:tasks.rados.rados.0.smithi138.stdout:4: finishing write tid 2 to smithi138138488-4 2023-06-27T12:38:06.634 INFO:tasks.rados.rados.0.smithi138.stdout:6: finishing write tid 3 to smithi138138488-6 2023-06-27T12:38:06.634 INFO:tasks.rados.rados.0.smithi138.stdout:9: finishing write tid 2 to smithi138138488-9 2023-06-27T12:38:06.634 INFO:tasks.rados.rados.0.smithi138.stdout:7: finishing write tid 2 to smithi138138488-7 2023-06-27T12:38:06.634 INFO:tasks.rados.rados.0.smithi138.stdout:9: finishing write tid 3 to smithi138138488-9 2023-06-27T12:38:06.639 INFO:tasks.rados.rados.0.smithi138.stdout:9: finishing write tid 4 to smithi138138488-9 2023-06-27T12:38:06.639 INFO:tasks.rados.rados.0.smithi138.stdout:9: oid 9 updating version 0 to 6 2023-06-27T12:38:06.639 INFO:tasks.rados.rados.0.smithi138.stdout:9: oid 9 version 6 is already newer than 6 2023-06-27T12:38:06.639 INFO:tasks.rados.rados.0.smithi138.stdout:9: oid 9 version 6 is already newer than 4 2023-06-27T12:38:06.639 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2023-06-27T12:38:06.640 INFO:tasks.rados.rados.0.smithi138.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2023-06-27T12:38:06.640 INFO:tasks.rados.rados.0.smithi138.stdout:9: done (15 left) 2023-06-27T12:38:06.640 INFO:tasks.rados.rados.0.smithi138.stdout:20: write initial oid 20 2023-06-27T12:38:06.640 INFO:tasks.rados.rados.0.smithi138.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2023-06-27T12:38:06.646 INFO:tasks.rados.rados.0.smithi138.stdout:20: writing smithi138138488-20 from 719333 to 1181770 tid 1 2023-06-27T12:38:06.653 INFO:tasks.rados.rados.0.smithi138.stdout:20: writing smithi138138488-20 from 1962099 to 2319333 tid 2 2023-06-27T12:38:06.653 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.653 INFO:tasks.rados.rados.0.smithi138.stdout:6: finishing write tid 4 to smithi138138488-6 2023-06-27T12:38:06.654 INFO:tasks.rados.rados.0.smithi138.stdout:6: oid 6 updating version 0 to 3 2023-06-27T12:38:06.654 INFO:tasks.rados.rados.0.smithi138.stdout:6: oid 6 version 3 is already newer than 3 2023-06-27T12:38:06.654 INFO:tasks.rados.rados.0.smithi138.stdout:6: oid 6 version 3 is already newer than 1 2023-06-27T12:38:06.654 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2023-06-27T12:38:06.654 INFO:tasks.rados.rados.0.smithi138.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2023-06-27T12:38:06.654 INFO:tasks.rados.rados.0.smithi138.stdout:6: done (15 left) 2023-06-27T12:38:06.654 INFO:tasks.rados.rados.0.smithi138.stdout:21: write initial oid 21 2023-06-27T12:38:06.655 INFO:tasks.rados.rados.0.smithi138.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2023-06-27T12:38:06.660 INFO:tasks.rados.rados.0.smithi138.stdout:21: writing smithi138138488-21 from 736140 to 1273826 tid 1 2023-06-27T12:38:06.666 INFO:tasks.rados.rados.0.smithi138.stdout:21: writing smithi138138488-21 from 2020581 to 2336140 tid 2 2023-06-27T12:38:06.667 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.667 INFO:tasks.rados.rados.0.smithi138.stdout:4: finishing write tid 3 to smithi138138488-4 2023-06-27T12:38:06.667 INFO:tasks.rados.rados.0.smithi138.stdout:13: finishing write tid 1 to smithi138138488-13 2023-06-27T12:38:06.667 INFO:tasks.rados.rados.0.smithi138.stdout:4: finishing write tid 4 to smithi138138488-4 2023-06-27T12:38:06.667 INFO:tasks.rados.rados.0.smithi138.stdout:4: oid 4 updating version 0 to 3 2023-06-27T12:38:06.667 INFO:tasks.rados.rados.0.smithi138.stdout:4: oid 4 version 3 is already newer than 1 2023-06-27T12:38:06.667 INFO:tasks.rados.rados.0.smithi138.stdout:4: oid 4 version 3 is already newer than 3 2023-06-27T12:38:06.668 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2023-06-27T12:38:06.668 INFO:tasks.rados.rados.0.smithi138.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2023-06-27T12:38:06.668 INFO:tasks.rados.rados.0.smithi138.stdout:8: finishing write tid 1 to smithi138138488-8 2023-06-27T12:38:06.668 INFO:tasks.rados.rados.0.smithi138.stdout:4: done (15 left) 2023-06-27T12:38:06.668 INFO:tasks.rados.rados.0.smithi138.stdout:22: write initial oid 22 2023-06-27T12:38:06.668 INFO:tasks.rados.rados.0.smithi138.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2023-06-27T12:38:06.674 INFO:tasks.rados.rados.0.smithi138.stdout:22: writing smithi138138488-22 from 752947 to 1365882 tid 1 2023-06-27T12:38:06.680 INFO:tasks.rados.rados.0.smithi138.stdout:22: writing smithi138138488-22 from 1962710 to 2352947 tid 2 2023-06-27T12:38:06.681 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.681 INFO:tasks.rados.rados.0.smithi138.stdout:7: finishing write tid 3 to smithi138138488-7 2023-06-27T12:38:06.681 INFO:tasks.rados.rados.0.smithi138.stdout:7: finishing write tid 4 to smithi138138488-7 2023-06-27T12:38:06.681 INFO:tasks.rados.rados.0.smithi138.stdout:7: oid 7 updating version 0 to 3 2023-06-27T12:38:06.681 INFO:tasks.rados.rados.0.smithi138.stdout:7: oid 7 version 3 is already newer than 3 2023-06-27T12:38:06.682 INFO:tasks.rados.rados.0.smithi138.stdout:7: oid 7 version 3 is already newer than 1 2023-06-27T12:38:06.682 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 3 (ObjNum 6 snap 0 seq_num 6) dirty exists 2023-06-27T12:38:06.682 INFO:tasks.rados.rados.0.smithi138.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2023-06-27T12:38:06.682 INFO:tasks.rados.rados.0.smithi138.stdout:17: finishing write tid 1 to smithi138138488-17 2023-06-27T12:38:06.682 INFO:tasks.rados.rados.0.smithi138.stdout:11: finishing write tid 2 to smithi138138488-11 2023-06-27T12:38:06.682 INFO:tasks.rados.rados.0.smithi138.stdout:7: done (15 left) 2023-06-27T12:38:06.682 INFO:tasks.rados.rados.0.smithi138.stdout:23: write initial oid 23 2023-06-27T12:38:06.682 INFO:tasks.rados.rados.0.smithi138.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2023-06-27T12:38:06.689 INFO:tasks.rados.rados.0.smithi138.stdout:23: writing smithi138138488-23 from 769754 to 1457938 tid 1 2023-06-27T12:38:06.694 INFO:tasks.rados.rados.0.smithi138.stdout:23: writing smithi138138488-23 from 2021192 to 2369754 tid 2 2023-06-27T12:38:06.695 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.695 INFO:tasks.rados.rados.0.smithi138.stdout:16: finishing write tid 1 to smithi138138488-16 2023-06-27T12:38:06.698 INFO:tasks.rados.rados.0.smithi138.stdout:10: finishing write tid 1 to smithi138138488-10 2023-06-27T12:38:06.700 INFO:tasks.rados.rados.0.smithi138.stdout:5: finishing write tid 1 to smithi138138488-5 2023-06-27T12:38:06.701 INFO:tasks.rados.rados.0.smithi138.stdout:8: finishing write tid 2 to smithi138138488-8 2023-06-27T12:38:06.703 INFO:tasks.rados.rados.0.smithi138.stdout:8: finishing write tid 3 to smithi138138488-8 2023-06-27T12:38:06.719 INFO:tasks.rados.rados.0.smithi138.stdout:11: finishing write tid 3 to smithi138138488-11 2023-06-27T12:38:06.720 INFO:tasks.rados.rados.0.smithi138.stdout:11: finishing write tid 4 to smithi138138488-11 2023-06-27T12:38:06.720 INFO:tasks.rados.rados.0.smithi138.stdout:11: oid 11 updating version 0 to 1 2023-06-27T12:38:06.720 INFO:tasks.rados.rados.0.smithi138.stdout:11: oid 11 updating version 1 to 3 2023-06-27T12:38:06.720 INFO:tasks.rados.rados.0.smithi138.stdout:11: oid 11 version 3 is already newer than 3 2023-06-27T12:38:06.720 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2023-06-27T12:38:06.720 INFO:tasks.rados.rados.0.smithi138.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2023-06-27T12:38:06.721 INFO:tasks.rados.rados.0.smithi138.stdout:11: done (15 left) 2023-06-27T12:38:06.721 INFO:tasks.rados.rados.0.smithi138.stdout:24: write initial oid 24 2023-06-27T12:38:06.721 INFO:tasks.rados.rados.0.smithi138.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2023-06-27T12:38:06.726 INFO:tasks.rados.rados.0.smithi138.stdout:24: writing smithi138138488-24 from 786561 to 1266347 tid 1 2023-06-27T12:38:06.733 INFO:tasks.rados.rados.0.smithi138.stdout:24: writing smithi138138488-24 from 1796027 to 2263691 tid 2 2023-06-27T12:38:06.733 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.733 INFO:tasks.rados.rados.0.smithi138.stdout:8: finishing write tid 4 to smithi138138488-8 2023-06-27T12:38:06.733 INFO:tasks.rados.rados.0.smithi138.stdout:8: oid 8 updating version 0 to 1 2023-06-27T12:38:06.733 INFO:tasks.rados.rados.0.smithi138.stdout:8: oid 8 updating version 1 to 3 2023-06-27T12:38:06.734 INFO:tasks.rados.rados.0.smithi138.stdout:8: oid 8 version 3 is already newer than 3 2023-06-27T12:38:06.734 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 8 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2023-06-27T12:38:06.734 INFO:tasks.rados.rados.0.smithi138.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2023-06-27T12:38:06.734 INFO:tasks.rados.rados.0.smithi138.stdout:8: done (15 left) 2023-06-27T12:38:06.734 INFO:tasks.rados.rados.0.smithi138.stdout:25: write initial oid 25 2023-06-27T12:38:06.734 INFO:tasks.rados.rados.0.smithi138.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2023-06-27T12:38:06.739 INFO:tasks.rados.rados.0.smithi138.stdout:25: writing smithi138138488-25 from 403368 to 958403 tid 1 2023-06-27T12:38:06.748 INFO:tasks.rados.rados.0.smithi138.stdout:25: writing smithi138138488-25 from 1454509 to 2182184 tid 2 2023-06-27T12:38:06.750 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.750 INFO:tasks.rados.rados.0.smithi138.stdout:17: finishing write tid 2 to smithi138138488-17 2023-06-27T12:38:06.750 INFO:tasks.rados.rados.0.smithi138.stdout:13: finishing write tid 2 to smithi138138488-13 2023-06-27T12:38:06.750 INFO:tasks.rados.rados.0.smithi138.stdout:13: finishing write tid 3 to smithi138138488-13 2023-06-27T12:38:06.751 INFO:tasks.rados.rados.0.smithi138.stdout:13: finishing write tid 4 to smithi138138488-13 2023-06-27T12:38:06.751 INFO:tasks.rados.rados.0.smithi138.stdout:13: oid 13 updating version 0 to 6 2023-06-27T12:38:06.751 INFO:tasks.rados.rados.0.smithi138.stdout:13: oid 13 version 6 is already newer than 6 2023-06-27T12:38:06.751 INFO:tasks.rados.rados.0.smithi138.stdout:13: oid 13 version 6 is already newer than 4 2023-06-27T12:38:06.751 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 6 (ObjNum 12 snap 0 seq_num 12) dirty exists 2023-06-27T12:38:06.751 INFO:tasks.rados.rados.0.smithi138.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2023-06-27T12:38:06.751 INFO:tasks.rados.rados.0.smithi138.stdout:14: finishing write tid 1 to smithi138138488-14 2023-06-27T12:38:06.752 INFO:tasks.rados.rados.0.smithi138.stdout:13: done (15 left) 2023-06-27T12:38:06.752 INFO:tasks.rados.rados.0.smithi138.stdout:26: write initial oid 26 2023-06-27T12:38:06.752 INFO:tasks.rados.rados.0.smithi138.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2023-06-27T12:38:06.754 INFO:tasks.rados.rados.0.smithi138.stdout:26: writing smithi138138488-26 from 420175 to 1050459 tid 1 2023-06-27T12:38:06.761 INFO:tasks.rados.rados.0.smithi138.stdout:26: writing smithi138138488-26 from 1796638 to 2267971 tid 2 2023-06-27T12:38:06.762 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.766 INFO:tasks.rados.rados.0.smithi138.stdout:16: finishing write tid 2 to smithi138138488-16 2023-06-27T12:38:06.768 INFO:tasks.rados.rados.0.smithi138.stdout:17: finishing write tid 3 to smithi138138488-17 2023-06-27T12:38:06.777 INFO:tasks.rados.rados.0.smithi138.stdout:15: finishing write tid 1 to smithi138138488-15 2023-06-27T12:38:06.790 INFO:tasks.rados.rados.0.smithi138.stdout:12: finishing write tid 1 to smithi138138488-12 2023-06-27T12:38:06.807 INFO:tasks.rados.rados.0.smithi138.stdout:16: finishing write tid 3 to smithi138138488-16 2023-06-27T12:38:06.807 INFO:tasks.rados.rados.0.smithi138.stdout:16: finishing write tid 4 to smithi138138488-16 2023-06-27T12:38:06.807 INFO:tasks.rados.rados.0.smithi138.stdout:16: oid 16 updating version 0 to 3 2023-06-27T12:38:06.807 INFO:tasks.rados.rados.0.smithi138.stdout:16: oid 16 version 3 is already newer than 3 2023-06-27T12:38:06.808 INFO:tasks.rados.rados.0.smithi138.stdout:16: oid 16 version 3 is already newer than 1 2023-06-27T12:38:06.808 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 3 (ObjNum 15 snap 0 seq_num 15) dirty exists 2023-06-27T12:38:06.808 INFO:tasks.rados.rados.0.smithi138.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2023-06-27T12:38:06.808 INFO:tasks.rados.rados.0.smithi138.stdout:10: finishing write tid 2 to smithi138138488-10 2023-06-27T12:38:06.808 INFO:tasks.rados.rados.0.smithi138.stdout:16: done (15 left) 2023-06-27T12:38:06.808 INFO:tasks.rados.rados.0.smithi138.stdout:27: write initial oid 27 2023-06-27T12:38:06.808 INFO:tasks.rados.rados.0.smithi138.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2023-06-27T12:38:06.814 INFO:tasks.rados.rados.0.smithi138.stdout:27: writing smithi138138488-27 from 436982 to 1142515 tid 1 2023-06-27T12:38:06.822 INFO:tasks.rados.rados.0.smithi138.stdout:27: writing smithi138138488-27 from 1855120 to 2436982 tid 2 2023-06-27T12:38:06.823 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.823 INFO:tasks.rados.rados.0.smithi138.stdout:17: finishing write tid 4 to smithi138138488-17 2023-06-27T12:38:06.823 INFO:tasks.rados.rados.0.smithi138.stdout:17: oid 17 updating version 0 to 3 2023-06-27T12:38:06.824 INFO:tasks.rados.rados.0.smithi138.stdout:17: oid 17 version 3 is already newer than 3 2023-06-27T12:38:06.824 INFO:tasks.rados.rados.0.smithi138.stdout:17: oid 17 version 3 is already newer than 1 2023-06-27T12:38:06.824 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 3 (ObjNum 16 snap 0 seq_num 16) dirty exists 2023-06-27T12:38:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2023-06-27T12:38:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:22: finishing write tid 1 to smithi138138488-22 2023-06-27T12:38:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:10: finishing write tid 3 to smithi138138488-10 2023-06-27T12:38:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:5: finishing write tid 2 to smithi138138488-5 2023-06-27T12:38:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:10: finishing write tid 4 to smithi138138488-10 2023-06-27T12:38:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:10: oid 10 updating version 0 to 4 2023-06-27T12:38:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:10: oid 10 updating version 4 to 6 2023-06-27T12:38:06.827 INFO:tasks.rados.rados.0.smithi138.stdout:10: oid 10 version 6 is already newer than 6 2023-06-27T12:38:06.827 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 6 (ObjNum 9 snap 0 seq_num 9) dirty exists 2023-06-27T12:38:06.827 INFO:tasks.rados.rados.0.smithi138.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2023-06-27T12:38:06.828 INFO:tasks.rados.rados.0.smithi138.stdout:5: finishing write tid 3 to smithi138138488-5 2023-06-27T12:38:06.828 INFO:tasks.rados.rados.0.smithi138.stdout:10: done (15 left) 2023-06-27T12:38:06.828 INFO:tasks.rados.rados.0.smithi138.stdout:17: done (14 left) 2023-06-27T12:38:06.828 INFO:tasks.rados.rados.0.smithi138.stdout:28: write initial oid 28 2023-06-27T12:38:06.829 INFO:tasks.rados.rados.0.smithi138.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2023-06-27T12:38:06.829 INFO:tasks.rados.rados.0.smithi138.stdout:28: writing smithi138138488-28 from 453789 to 1234571 tid 1 2023-06-27T12:38:06.838 INFO:tasks.rados.rados.0.smithi138.stdout:28: writing smithi138138488-28 from 1913602 to 2388604 tid 2 2023-06-27T12:38:06.838 INFO:tasks.rados.rados.0.smithi138.stdout:29: write initial oid 29 2023-06-27T12:38:06.839 INFO:tasks.rados.rados.0.smithi138.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2023-06-27T12:38:06.845 INFO:tasks.rados.rados.0.smithi138.stdout:29: writing smithi138138488-29 from 470596 to 926627 tid 1 2023-06-27T12:38:06.856 INFO:tasks.rados.rados.0.smithi138.stdout:29: writing smithi138138488-29 from 1572084 to 2190744 tid 2 2023-06-27T12:38:06.857 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.857 INFO:tasks.rados.rados.0.smithi138.stdout:5: finishing write tid 4 to smithi138138488-5 2023-06-27T12:38:06.857 INFO:tasks.rados.rados.0.smithi138.stdout:5: oid 5 updating version 0 to 4 2023-06-27T12:38:06.857 INFO:tasks.rados.rados.0.smithi138.stdout:5: oid 5 updating version 4 to 6 2023-06-27T12:38:06.857 INFO:tasks.rados.rados.0.smithi138.stdout:5: oid 5 version 6 is already newer than 6 2023-06-27T12:38:06.857 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 5 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2023-06-27T12:38:06.857 INFO:tasks.rados.rados.0.smithi138.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2023-06-27T12:38:06.858 INFO:tasks.rados.rados.0.smithi138.stdout:14: finishing write tid 2 to smithi138138488-14 2023-06-27T12:38:06.858 INFO:tasks.rados.rados.0.smithi138.stdout:5: done (15 left) 2023-06-27T12:38:06.858 INFO:tasks.rados.rados.0.smithi138.stdout:30: write initial oid 30 2023-06-27T12:38:06.858 INFO:tasks.rados.rados.0.smithi138.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2023-06-27T12:38:06.862 INFO:tasks.rados.rados.0.smithi138.stdout:30: writing smithi138138488-30 from 487403 to 1018683 tid 1 2023-06-27T12:38:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:30: writing smithi138138488-30 from 1514213 to 1992884 tid 2 2023-06-27T12:38:06.869 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:23: finishing write tid 1 to smithi138138488-23 2023-06-27T12:38:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:14: finishing write tid 3 to smithi138138488-14 2023-06-27T12:38:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:14: finishing write tid 4 to smithi138138488-14 2023-06-27T12:38:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:14: oid 14 updating version 0 to 7 2023-06-27T12:38:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:14: oid 14 updating version 7 to 9 2023-06-27T12:38:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:14: oid 14 version 9 is already newer than 9 2023-06-27T12:38:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 9 (ObjNum 13 snap 0 seq_num 13) dirty exists 2023-06-27T12:38:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2023-06-27T12:38:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:15: finishing write tid 2 to smithi138138488-15 2023-06-27T12:38:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:25: finishing write tid 1 to smithi138138488-25 2023-06-27T12:38:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:14: done (15 left) 2023-06-27T12:38:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:31: write initial oid 31 2023-06-27T12:38:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2023-06-27T12:38:06.877 INFO:tasks.rados.rados.0.smithi138.stdout:31: writing smithi138138488-31 from 504210 to 1110739 tid 1 2023-06-27T12:38:06.887 INFO:tasks.rados.rados.0.smithi138.stdout:31: writing smithi138138488-31 from 1572695 to 2195024 tid 2 2023-06-27T12:38:06.888 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.888 INFO:tasks.rados.rados.0.smithi138.stdout:15: finishing write tid 3 to smithi138138488-15 2023-06-27T12:38:06.888 INFO:tasks.rados.rados.0.smithi138.stdout:15: finishing write tid 4 to smithi138138488-15 2023-06-27T12:38:06.888 INFO:tasks.rados.rados.0.smithi138.stdout:15: oid 15 updating version 0 to 4 2023-06-27T12:38:06.888 INFO:tasks.rados.rados.0.smithi138.stdout:15: oid 15 updating version 4 to 6 2023-06-27T12:38:06.888 INFO:tasks.rados.rados.0.smithi138.stdout:15: oid 15 version 6 is already newer than 6 2023-06-27T12:38:06.888 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 6 (ObjNum 14 snap 0 seq_num 14) dirty exists 2023-06-27T12:38:06.889 INFO:tasks.rados.rados.0.smithi138.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2023-06-27T12:38:06.889 INFO:tasks.rados.rados.0.smithi138.stdout:12: finishing write tid 2 to smithi138138488-12 2023-06-27T12:38:06.889 INFO:tasks.rados.rados.0.smithi138.stdout:26: finishing write tid 1 to smithi138138488-26 2023-06-27T12:38:06.889 INFO:tasks.rados.rados.0.smithi138.stdout:20: finishing write tid 1 to smithi138138488-20 2023-06-27T12:38:06.889 INFO:tasks.rados.rados.0.smithi138.stdout:15: done (15 left) 2023-06-27T12:38:06.889 INFO:tasks.rados.rados.0.smithi138.stdout:32: write initial oid 32 2023-06-27T12:38:06.889 INFO:tasks.rados.rados.0.smithi138.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2023-06-27T12:38:06.897 INFO:tasks.rados.rados.0.smithi138.stdout:32: writing smithi138138488-32 from 521017 to 1319148 tid 1 2023-06-27T12:38:06.906 INFO:tasks.rados.rados.0.smithi138.stdout:32: writing smithi138138488-32 from 1747530 to 2229870 tid 2 2023-06-27T12:38:06.906 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.906 INFO:tasks.rados.rados.0.smithi138.stdout:22: finishing write tid 2 to smithi138138488-22 2023-06-27T12:38:06.906 INFO:tasks.rados.rados.0.smithi138.stdout:12: finishing write tid 3 to smithi138138488-12 2023-06-27T12:38:06.906 INFO:tasks.rados.rados.0.smithi138.stdout:22: finishing write tid 3 to smithi138138488-22 2023-06-27T12:38:06.906 INFO:tasks.rados.rados.0.smithi138.stdout:22: finishing write tid 4 to smithi138138488-22 2023-06-27T12:38:06.907 INFO:tasks.rados.rados.0.smithi138.stdout:22: oid 22 updating version 0 to 3 2023-06-27T12:38:06.907 INFO:tasks.rados.rados.0.smithi138.stdout:22: oid 22 version 3 is already newer than 3 2023-06-27T12:38:06.907 INFO:tasks.rados.rados.0.smithi138.stdout:22: oid 22 version 3 is already newer than 1 2023-06-27T12:38:06.907 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 3 (ObjNum 21 snap 0 seq_num 21) dirty exists 2023-06-27T12:38:06.907 INFO:tasks.rados.rados.0.smithi138.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2023-06-27T12:38:06.907 INFO:tasks.rados.rados.0.smithi138.stdout:23: finishing write tid 2 to smithi138138488-23 2023-06-27T12:38:06.907 INFO:tasks.rados.rados.0.smithi138.stdout:24: finishing write tid 1 to smithi138138488-24 2023-06-27T12:38:06.907 INFO:tasks.rados.rados.0.smithi138.stdout:22: done (15 left) 2023-06-27T12:38:06.908 INFO:tasks.rados.rados.0.smithi138.stdout:33: write initial oid 33 2023-06-27T12:38:06.908 INFO:tasks.rados.rados.0.smithi138.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2023-06-27T12:38:06.913 INFO:tasks.rados.rados.0.smithi138.stdout:33: writing smithi138138488-33 from 537824 to 1011204 tid 1 2023-06-27T12:38:06.925 INFO:tasks.rados.rados.0.smithi138.stdout:33: writing smithi138138488-33 from 1806012 to 2432010 tid 2 2023-06-27T12:38:06.925 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.925 INFO:tasks.rados.rados.0.smithi138.stdout:12: finishing write tid 4 to smithi138138488-12 2023-06-27T12:38:06.926 INFO:tasks.rados.rados.0.smithi138.stdout:12: oid 12 updating version 0 to 4 2023-06-27T12:38:06.926 INFO:tasks.rados.rados.0.smithi138.stdout:12: oid 12 updating version 4 to 6 2023-06-27T12:38:06.926 INFO:tasks.rados.rados.0.smithi138.stdout:12: oid 12 version 6 is already newer than 6 2023-06-27T12:38:06.926 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2023-06-27T12:38:06.926 INFO:tasks.rados.rados.0.smithi138.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2023-06-27T12:38:06.926 INFO:tasks.rados.rados.0.smithi138.stdout:23: finishing write tid 3 to smithi138138488-23 2023-06-27T12:38:06.926 INFO:tasks.rados.rados.0.smithi138.stdout:23: finishing write tid 4 to smithi138138488-23 2023-06-27T12:38:06.926 INFO:tasks.rados.rados.0.smithi138.stdout:23: oid 23 updating version 0 to 6 2023-06-27T12:38:06.927 INFO:tasks.rados.rados.0.smithi138.stdout:23: oid 23 version 6 is already newer than 4 2023-06-27T12:38:06.927 INFO:tasks.rados.rados.0.smithi138.stdout:23: oid 23 version 6 is already newer than 6 2023-06-27T12:38:06.927 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 6 (ObjNum 22 snap 0 seq_num 22) dirty exists 2023-06-27T12:38:06.927 INFO:tasks.rados.rados.0.smithi138.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2023-06-27T12:38:06.927 INFO:tasks.rados.rados.0.smithi138.stdout:20: finishing write tid 2 to smithi138138488-20 2023-06-27T12:38:06.927 INFO:tasks.rados.rados.0.smithi138.stdout:12: done (15 left) 2023-06-27T12:38:06.927 INFO:tasks.rados.rados.0.smithi138.stdout:23: done (14 left) 2023-06-27T12:38:06.928 INFO:tasks.rados.rados.0.smithi138.stdout:34: write initial oid 34 2023-06-27T12:38:06.928 INFO:tasks.rados.rados.0.smithi138.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2023-06-27T12:38:06.934 INFO:tasks.rados.rados.0.smithi138.stdout:34: writing smithi138138488-34 from 554631 to 1103260 tid 1 2023-06-27T12:38:06.943 INFO:tasks.rados.rados.0.smithi138.stdout:34: writing smithi138138488-34 from 1748141 to 2234150 tid 2 2023-06-27T12:38:06.943 INFO:tasks.rados.rados.0.smithi138.stdout:35: write initial oid 35 2023-06-27T12:38:06.943 INFO:tasks.rados.rados.0.smithi138.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2023-06-27T12:38:06.952 INFO:tasks.rados.rados.0.smithi138.stdout:35: writing smithi138138488-35 from 571438 to 1195316 tid 1 2023-06-27T12:38:06.962 INFO:tasks.rados.rados.0.smithi138.stdout:35: writing smithi138138488-35 from 1806623 to 2436290 tid 2 2023-06-27T12:38:06.963 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.963 INFO:tasks.rados.rados.0.smithi138.stdout:20: finishing write tid 3 to smithi138138488-20 2023-06-27T12:38:06.963 INFO:tasks.rados.rados.0.smithi138.stdout:20: finishing write tid 4 to smithi138138488-20 2023-06-27T12:38:06.963 INFO:tasks.rados.rados.0.smithi138.stdout:20: oid 20 updating version 0 to 6 2023-06-27T12:38:06.963 INFO:tasks.rados.rados.0.smithi138.stdout:20: oid 20 version 6 is already newer than 6 2023-06-27T12:38:06.963 INFO:tasks.rados.rados.0.smithi138.stdout:20: oid 20 version 6 is already newer than 4 2023-06-27T12:38:06.963 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 6 (ObjNum 19 snap 0 seq_num 19) dirty exists 2023-06-27T12:38:06.964 INFO:tasks.rados.rados.0.smithi138.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2023-06-27T12:38:06.964 INFO:tasks.rados.rados.0.smithi138.stdout:19: finishing write tid 1 to smithi138138488-19 2023-06-27T12:38:06.964 INFO:tasks.rados.rados.0.smithi138.stdout:20: done (15 left) 2023-06-27T12:38:06.964 INFO:tasks.rados.rados.0.smithi138.stdout:36: write initial oid 36 2023-06-27T12:38:06.964 INFO:tasks.rados.rados.0.smithi138.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2023-06-27T12:38:06.973 INFO:tasks.rados.rados.0.smithi138.stdout:36: writing smithi138138488-36 from 588245 to 1287372 tid 1 2023-06-27T12:38:06.982 INFO:tasks.rados.rados.0.smithi138.stdout:36: writing smithi138138488-36 from 1865105 to 2354783 tid 2 2023-06-27T12:38:06.982 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:06.982 INFO:tasks.rados.rados.0.smithi138.stdout:25: finishing write tid 2 to smithi138138488-25 2023-06-27T12:38:06.982 INFO:tasks.rados.rados.0.smithi138.stdout:25: finishing write tid 3 to smithi138138488-25 2023-06-27T12:38:06.983 INFO:tasks.rados.rados.0.smithi138.stdout:25: finishing write tid 4 to smithi138138488-25 2023-06-27T12:38:06.983 INFO:tasks.rados.rados.0.smithi138.stdout:25: oid 25 updating version 0 to 3 2023-06-27T12:38:06.983 INFO:tasks.rados.rados.0.smithi138.stdout:25: oid 25 version 3 is already newer than 3 2023-06-27T12:38:06.983 INFO:tasks.rados.rados.0.smithi138.stdout:25: oid 25 version 3 is already newer than 1 2023-06-27T12:38:06.983 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 3 (ObjNum 24 snap 0 seq_num 24) dirty exists 2023-06-27T12:38:06.983 INFO:tasks.rados.rados.0.smithi138.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2023-06-27T12:38:06.983 INFO:tasks.rados.rados.0.smithi138.stdout:25: done (15 left) 2023-06-27T12:38:06.984 INFO:tasks.rados.rados.0.smithi138.stdout:37: write initial oid 37 2023-06-27T12:38:06.984 INFO:tasks.rados.rados.0.smithi138.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2023-06-27T12:38:06.993 INFO:tasks.rados.rados.0.smithi138.stdout:37: writing smithi138138488-37 from 605052 to 1379428 tid 1 2023-06-27T12:38:07.003 INFO:tasks.rados.rados.0.smithi138.stdout:37: writing smithi138138488-37 from 1923587 to 2556923 tid 2 2023-06-27T12:38:07.003 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.003 INFO:tasks.rados.rados.0.smithi138.stdout:28: finishing write tid 1 to smithi138138488-28 2023-06-27T12:38:07.003 INFO:tasks.rados.rados.0.smithi138.stdout:26: finishing write tid 2 to smithi138138488-26 2023-06-27T12:38:07.004 INFO:tasks.rados.rados.0.smithi138.stdout:26: finishing write tid 3 to smithi138138488-26 2023-06-27T12:38:07.004 INFO:tasks.rados.rados.0.smithi138.stdout:26: finishing write tid 4 to smithi138138488-26 2023-06-27T12:38:07.004 INFO:tasks.rados.rados.0.smithi138.stdout:26: oid 26 updating version 0 to 4 2023-06-27T12:38:07.005 INFO:tasks.rados.rados.0.smithi138.stdout:26: oid 26 updating version 4 to 6 2023-06-27T12:38:07.005 INFO:tasks.rados.rados.0.smithi138.stdout:26: oid 26 version 6 is already newer than 6 2023-06-27T12:38:07.005 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 6 (ObjNum 25 snap 0 seq_num 25) dirty exists 2023-06-27T12:38:07.005 INFO:tasks.rados.rados.0.smithi138.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2023-06-27T12:38:07.005 INFO:tasks.rados.rados.0.smithi138.stdout:26: done (15 left) 2023-06-27T12:38:07.005 INFO:tasks.rados.rados.0.smithi138.stdout:38: write initial oid 38 2023-06-27T12:38:07.005 INFO:tasks.rados.rados.0.smithi138.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2023-06-27T12:38:07.011 INFO:tasks.rados.rados.0.smithi138.stdout:38: writing smithi138138488-38 from 621859 to 1071484 tid 1 2023-06-27T12:38:07.024 INFO:tasks.rados.rados.0.smithi138.stdout:38: writing smithi138138488-38 from 1865716 to 2621859 tid 2 2023-06-27T12:38:07.024 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.024 INFO:tasks.rados.rados.0.smithi138.stdout:27: finishing write tid 1 to smithi138138488-27 2023-06-27T12:38:07.025 INFO:tasks.rados.rados.0.smithi138.stdout:18: finishing write tid 1 to smithi138138488-18 2023-06-27T12:38:07.025 INFO:tasks.rados.rados.0.smithi138.stdout:30: finishing write tid 1 to smithi138138488-30 2023-06-27T12:38:07.049 INFO:tasks.rados.rados.0.smithi138.stdout:24: finishing write tid 2 to smithi138138488-24 2023-06-27T12:38:07.056 INFO:tasks.rados.rados.0.smithi138.stdout:32: finishing write tid 1 to smithi138138488-32 2023-06-27T12:38:07.084 INFO:tasks.rados.rados.0.smithi138.stdout:29: finishing write tid 1 to smithi138138488-29 2023-06-27T12:38:07.086 INFO:tasks.rados.rados.0.smithi138.stdout:19: finishing write tid 2 to smithi138138488-19 2023-06-27T12:38:07.090 INFO:tasks.rados.rados.0.smithi138.stdout:24: finishing write tid 3 to smithi138138488-24 2023-06-27T12:38:07.091 INFO:tasks.rados.rados.0.smithi138.stdout:24: finishing write tid 4 to smithi138138488-24 2023-06-27T12:38:07.091 INFO:tasks.rados.rados.0.smithi138.stdout:24: oid 24 updating version 0 to 6 2023-06-27T12:38:07.091 INFO:tasks.rados.rados.0.smithi138.stdout:24: oid 24 version 6 is already newer than 6 2023-06-27T12:38:07.091 INFO:tasks.rados.rados.0.smithi138.stdout:24: oid 24 version 6 is already newer than 4 2023-06-27T12:38:07.091 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 6 (ObjNum 23 snap 0 seq_num 23) dirty exists 2023-06-27T12:38:07.091 INFO:tasks.rados.rados.0.smithi138.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2023-06-27T12:38:07.091 INFO:tasks.rados.rados.0.smithi138.stdout:24: done (15 left) 2023-06-27T12:38:07.091 INFO:tasks.rados.rados.0.smithi138.stdout:39: write initial oid 39 2023-06-27T12:38:07.092 INFO:tasks.rados.rados.0.smithi138.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2023-06-27T12:38:07.099 INFO:tasks.rados.rados.0.smithi138.stdout:39: writing smithi138138488-39 from 638666 to 1163540 tid 1 2023-06-27T12:38:07.111 INFO:tasks.rados.rados.0.smithi138.stdout:39: writing smithi138138488-39 from 1924198 to 2561203 tid 2 2023-06-27T12:38:07.111 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.111 INFO:tasks.rados.rados.0.smithi138.stdout:27: finishing write tid 2 to smithi138138488-27 2023-06-27T12:38:07.112 INFO:tasks.rados.rados.0.smithi138.stdout:27: finishing write tid 3 to smithi138138488-27 2023-06-27T12:38:07.112 INFO:tasks.rados.rados.0.smithi138.stdout:27: finishing write tid 4 to smithi138138488-27 2023-06-27T12:38:07.112 INFO:tasks.rados.rados.0.smithi138.stdout:27: oid 27 updating version 0 to 9 2023-06-27T12:38:07.112 INFO:tasks.rados.rados.0.smithi138.stdout:27: oid 27 version 9 is already newer than 7 2023-06-27T12:38:07.112 INFO:tasks.rados.rados.0.smithi138.stdout:27: oid 27 version 9 is already newer than 9 2023-06-27T12:38:07.112 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 9 (ObjNum 26 snap 0 seq_num 26) dirty exists 2023-06-27T12:38:07.112 INFO:tasks.rados.rados.0.smithi138.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2023-06-27T12:38:07.113 INFO:tasks.rados.rados.0.smithi138.stdout:19: finishing write tid 3 to smithi138138488-19 2023-06-27T12:38:07.113 INFO:tasks.rados.rados.0.smithi138.stdout:19: finishing write tid 4 to smithi138138488-19 2023-06-27T12:38:07.113 INFO:tasks.rados.rados.0.smithi138.stdout:19: oid 19 updating version 0 to 12 2023-06-27T12:38:07.113 INFO:tasks.rados.rados.0.smithi138.stdout:19: oid 19 version 12 is already newer than 12 2023-06-27T12:38:07.113 INFO:tasks.rados.rados.0.smithi138.stdout:19: oid 19 version 12 is already newer than 10 2023-06-27T12:38:07.113 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 12 (ObjNum 18 snap 0 seq_num 18) dirty exists 2023-06-27T12:38:07.113 INFO:tasks.rados.rados.0.smithi138.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2023-06-27T12:38:07.113 INFO:tasks.rados.rados.0.smithi138.stdout:19: done (15 left) 2023-06-27T12:38:07.114 INFO:tasks.rados.rados.0.smithi138.stdout:27: done (14 left) 2023-06-27T12:38:07.114 INFO:tasks.rados.rados.0.smithi138.stdout:40: write initial oid 40 2023-06-27T12:38:07.114 INFO:tasks.rados.rados.0.smithi138.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2023-06-27T12:38:07.121 INFO:tasks.rados.rados.0.smithi138.stdout:40: writing smithi138138488-40 from 655473 to 1371949 tid 1 2023-06-27T12:38:07.124 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:07.131 INFO:tasks.rados.rados.0.smithi138.stdout:40: writing smithi138138488-40 from 2099033 to 2655473 tid 2 2023-06-27T12:38:07.131 INFO:tasks.rados.rados.0.smithi138.stdout:41: write initial oid 41 2023-06-27T12:38:07.131 INFO:tasks.rados.rados.0.smithi138.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2023-06-27T12:38:07.143 INFO:tasks.rados.rados.0.smithi138.stdout:41: writing smithi138138488-41 from 672280 to 1464005 tid 1 2023-06-27T12:38:07.153 INFO:tasks.rados.rados.0.smithi138.stdout:41: writing smithi138138488-41 from 2157515 to 2672280 tid 2 2023-06-27T12:38:07.153 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.153 INFO:tasks.rados.rados.0.smithi138.stdout:33: finishing write tid 1 to smithi138138488-33 2023-06-27T12:38:07.153 INFO:tasks.rados.rados.0.smithi138.stdout:30: finishing write tid 2 to smithi138138488-30 2023-06-27T12:38:07.153 INFO:tasks.rados.rados.0.smithi138.stdout:28: finishing write tid 2 to smithi138138488-28 2023-06-27T12:38:07.153 INFO:tasks.rados.rados.0.smithi138.stdout:18: finishing write tid 2 to smithi138138488-18 2023-06-27T12:38:07.157 INFO:tasks.rados.rados.0.smithi138.stdout:18: finishing write tid 3 to smithi138138488-18 2023-06-27T12:38:07.161 INFO:tasks.rados.rados.0.smithi138.stdout:30: finishing write tid 3 to smithi138138488-30 2023-06-27T12:38:07.162 INFO:tasks.rados.rados.0.smithi138.stdout:30: finishing write tid 4 to smithi138138488-30 2023-06-27T12:38:07.162 INFO:tasks.rados.rados.0.smithi138.stdout:30: oid 30 updating version 0 to 9 2023-06-27T12:38:07.162 INFO:tasks.rados.rados.0.smithi138.stdout:30: oid 30 version 9 is already newer than 9 2023-06-27T12:38:07.162 INFO:tasks.rados.rados.0.smithi138.stdout:30: oid 30 version 9 is already newer than 7 2023-06-27T12:38:07.162 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 9 (ObjNum 29 snap 0 seq_num 29) dirty exists 2023-06-27T12:38:07.162 INFO:tasks.rados.rados.0.smithi138.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2023-06-27T12:38:07.162 INFO:tasks.rados.rados.0.smithi138.stdout:30: done (15 left) 2023-06-27T12:38:07.163 INFO:tasks.rados.rados.0.smithi138.stdout:42: write initial oid 42 2023-06-27T12:38:07.163 INFO:tasks.rados.rados.0.smithi138.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2023-06-27T12:38:07.170 INFO:tasks.rados.rados.0.smithi138.stdout:42: writing smithi138138488-42 from 689087 to 1156061 tid 1 2023-06-27T12:38:07.182 INFO:tasks.rados.rados.0.smithi138.stdout:42: writing smithi138138488-42 from 1699644 to 2483976 tid 2 2023-06-27T12:38:07.182 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.182 INFO:tasks.rados.rados.0.smithi138.stdout:18: finishing write tid 4 to smithi138138488-18 2023-06-27T12:38:07.182 INFO:tasks.rados.rados.0.smithi138.stdout:18: oid 18 updating version 0 to 7 2023-06-27T12:38:07.182 INFO:tasks.rados.rados.0.smithi138.stdout:18: oid 18 updating version 7 to 9 2023-06-27T12:38:07.183 INFO:tasks.rados.rados.0.smithi138.stdout:18: oid 18 version 9 is already newer than 9 2023-06-27T12:38:07.183 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 9 (ObjNum 17 snap 0 seq_num 17) dirty exists 2023-06-27T12:38:07.183 INFO:tasks.rados.rados.0.smithi138.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2023-06-27T12:38:07.183 INFO:tasks.rados.rados.0.smithi138.stdout:18: done (15 left) 2023-06-27T12:38:07.183 INFO:tasks.rados.rados.0.smithi138.stdout:43: write initial oid 43 2023-06-27T12:38:07.183 INFO:tasks.rados.rados.0.smithi138.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2023-06-27T12:38:07.191 INFO:tasks.rados.rados.0.smithi138.stdout:43: writing smithi138138488-43 from 705894 to 1248117 tid 1 2023-06-27T12:38:07.202 INFO:tasks.rados.rados.0.smithi138.stdout:43: writing smithi138138488-43 from 1758126 to 2402469 tid 2 2023-06-27T12:38:07.202 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.202 INFO:tasks.rados.rados.0.smithi138.stdout:28: finishing write tid 3 to smithi138138488-28 2023-06-27T12:38:07.202 INFO:tasks.rados.rados.0.smithi138.stdout:28: finishing write tid 4 to smithi138138488-28 2023-06-27T12:38:07.202 INFO:tasks.rados.rados.0.smithi138.stdout:28: oid 28 updating version 0 to 9 2023-06-27T12:38:07.202 INFO:tasks.rados.rados.0.smithi138.stdout:28: oid 28 version 9 is already newer than 7 2023-06-27T12:38:07.203 INFO:tasks.rados.rados.0.smithi138.stdout:28: oid 28 version 9 is already newer than 9 2023-06-27T12:38:07.203 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 9 (ObjNum 27 snap 0 seq_num 27) dirty exists 2023-06-27T12:38:07.203 INFO:tasks.rados.rados.0.smithi138.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2023-06-27T12:38:07.203 INFO:tasks.rados.rados.0.smithi138.stdout:32: finishing write tid 2 to smithi138138488-32 2023-06-27T12:38:07.203 INFO:tasks.rados.rados.0.smithi138.stdout:32: finishing write tid 3 to smithi138138488-32 2023-06-27T12:38:07.203 INFO:tasks.rados.rados.0.smithi138.stdout:32: finishing write tid 4 to smithi138138488-32 2023-06-27T12:38:07.203 INFO:tasks.rados.rados.0.smithi138.stdout:32: oid 32 updating version 0 to 9 2023-06-27T12:38:07.203 INFO:tasks.rados.rados.0.smithi138.stdout:32: oid 32 version 9 is already newer than 9 2023-06-27T12:38:07.204 INFO:tasks.rados.rados.0.smithi138.stdout:32: oid 32 version 9 is already newer than 7 2023-06-27T12:38:07.204 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 9 (ObjNum 31 snap 0 seq_num 31) dirty exists 2023-06-27T12:38:07.204 INFO:tasks.rados.rados.0.smithi138.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2023-06-27T12:38:07.204 INFO:tasks.rados.rados.0.smithi138.stdout:29: finishing write tid 2 to smithi138138488-29 2023-06-27T12:38:07.204 INFO:tasks.rados.rados.0.smithi138.stdout:28: done (15 left) 2023-06-27T12:38:07.204 INFO:tasks.rados.rados.0.smithi138.stdout:32: done (14 left) 2023-06-27T12:38:07.204 INFO:tasks.rados.rados.0.smithi138.stdout:44: write initial oid 44 2023-06-27T12:38:07.205 INFO:tasks.rados.rados.0.smithi138.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2023-06-27T12:38:07.211 INFO:tasks.rados.rados.0.smithi138.stdout:44: writing smithi138138488-44 from 722701 to 1340173 tid 1 2023-06-27T12:38:07.222 INFO:tasks.rados.rados.0.smithi138.stdout:44: writing smithi138138488-44 from 1816608 to 2604609 tid 2 2023-06-27T12:38:07.222 INFO:tasks.rados.rados.0.smithi138.stdout:45: write initial oid 45 2023-06-27T12:38:07.222 INFO:tasks.rados.rados.0.smithi138.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2023-06-27T12:38:07.233 INFO:tasks.rados.rados.0.smithi138.stdout:45: writing smithi138138488-45 from 739508 to 1432229 tid 1 2023-06-27T12:38:07.243 INFO:tasks.rados.rados.0.smithi138.stdout:45: writing smithi138138488-45 from 1875090 to 2523102 tid 2 2023-06-27T12:38:07.243 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.243 INFO:tasks.rados.rados.0.smithi138.stdout:29: finishing write tid 3 to smithi138138488-29 2023-06-27T12:38:07.243 INFO:tasks.rados.rados.0.smithi138.stdout:29: finishing write tid 4 to smithi138138488-29 2023-06-27T12:38:07.243 INFO:tasks.rados.rados.0.smithi138.stdout:29: oid 29 updating version 0 to 6 2023-06-27T12:38:07.243 INFO:tasks.rados.rados.0.smithi138.stdout:29: oid 29 version 6 is already newer than 4 2023-06-27T12:38:07.244 INFO:tasks.rados.rados.0.smithi138.stdout:29: oid 29 version 6 is already newer than 6 2023-06-27T12:38:07.244 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 6 (ObjNum 28 snap 0 seq_num 28) dirty exists 2023-06-27T12:38:07.244 INFO:tasks.rados.rados.0.smithi138.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2023-06-27T12:38:07.244 INFO:tasks.rados.rados.0.smithi138.stdout:33: finishing write tid 2 to smithi138138488-33 2023-06-27T12:38:07.244 INFO:tasks.rados.rados.0.smithi138.stdout:39: finishing write tid 1 to smithi138138488-39 2023-06-27T12:38:07.244 INFO:tasks.rados.rados.0.smithi138.stdout:31: finishing write tid 1 to smithi138138488-31 2023-06-27T12:38:07.244 INFO:tasks.rados.rados.0.smithi138.stdout:21: finishing write tid 1 to smithi138138488-21 2023-06-27T12:38:07.245 INFO:tasks.rados.rados.0.smithi138.stdout:29: done (15 left) 2023-06-27T12:38:07.245 INFO:tasks.rados.rados.0.smithi138.stdout:46: write initial oid 46 2023-06-27T12:38:07.245 INFO:tasks.rados.rados.0.smithi138.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2023-06-27T12:38:07.254 INFO:tasks.rados.rados.0.smithi138.stdout:46: writing smithi138138488-46 from 756315 to 1524285 tid 1 2023-06-27T12:38:07.265 INFO:tasks.rados.rados.0.smithi138.stdout:46: writing smithi138138488-46 from 1933572 to 2725242 tid 2 2023-06-27T12:38:07.266 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.270 INFO:tasks.rados.rados.0.smithi138.stdout:33: finishing write tid 3 to smithi138138488-33 2023-06-27T12:38:07.270 INFO:tasks.rados.rados.0.smithi138.stdout:33: finishing write tid 4 to smithi138138488-33 2023-06-27T12:38:07.271 INFO:tasks.rados.rados.0.smithi138.stdout:33: oid 33 updating version 0 to 1 2023-06-27T12:38:07.271 INFO:tasks.rados.rados.0.smithi138.stdout:33: oid 33 updating version 1 to 3 2023-06-27T12:38:07.271 INFO:tasks.rados.rados.0.smithi138.stdout:33: oid 33 version 3 is already newer than 3 2023-06-27T12:38:07.271 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 3 (ObjNum 32 snap 0 seq_num 32) dirty exists 2023-06-27T12:38:07.271 INFO:tasks.rados.rados.0.smithi138.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2023-06-27T12:38:07.271 INFO:tasks.rados.rados.0.smithi138.stdout:33: done (15 left) 2023-06-27T12:38:07.271 INFO:tasks.rados.rados.0.smithi138.stdout:47: write initial oid 47 2023-06-27T12:38:07.272 INFO:tasks.rados.rados.0.smithi138.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2023-06-27T12:38:07.275 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:07.270+0000 7fd9f4bf3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:07.278 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:07.275+0000 7fd9f4bf3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:07.279 INFO:tasks.rados.rados.0.smithi138.stdout:47: writing smithi138138488-47 from 773122 to 1332694 tid 1 2023-06-27T12:38:07.287 INFO:tasks.rados.rados.0.smithi138.stdout:47: writing smithi138138488-47 from 1992054 to 2643735 tid 2 2023-06-27T12:38:07.288 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.288 INFO:tasks.rados.rados.0.smithi138.stdout:35: finishing write tid 1 to smithi138138488-35 2023-06-27T12:38:07.288 INFO:tasks.rados.rados.0.smithi138.stdout:41: finishing write tid 1 to smithi138138488-41 2023-06-27T12:38:07.294 INFO:tasks.rados.rados.0.smithi138.stdout:38: finishing write tid 1 to smithi138138488-38 2023-06-27T12:38:07.296 INFO:tasks.rados.rados.0.smithi138.stdout:36: finishing write tid 1 to smithi138138488-36 2023-06-27T12:38:07.339 INFO:tasks.rados.rados.0.smithi138.stdout:35: finishing write tid 2 to smithi138138488-35 2023-06-27T12:38:07.340 INFO:tasks.rados.rados.0.smithi138.stdout:21: finishing write tid 2 to smithi138138488-21 2023-06-27T12:38:07.341 INFO:tasks.rados.rados.0.smithi138.stdout:35: finishing write tid 3 to smithi138138488-35 2023-06-27T12:38:07.343 INFO:tasks.rados.rados.0.smithi138.stdout:46: finishing write tid 1 to smithi138138488-46 2023-06-27T12:38:07.350 INFO:tasks.rados.rados.0.smithi138.stdout:35: finishing write tid 4 to smithi138138488-35 2023-06-27T12:38:07.350 INFO:tasks.rados.rados.0.smithi138.stdout:35: oid 35 updating version 0 to 12 2023-06-27T12:38:07.351 INFO:tasks.rados.rados.0.smithi138.stdout:35: oid 35 version 12 is already newer than 12 2023-06-27T12:38:07.351 INFO:tasks.rados.rados.0.smithi138.stdout:35: oid 35 version 12 is already newer than 10 2023-06-27T12:38:07.351 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 12 (ObjNum 34 snap 0 seq_num 34) dirty exists 2023-06-27T12:38:07.351 INFO:tasks.rados.rados.0.smithi138.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2023-06-27T12:38:07.351 INFO:tasks.rados.rados.0.smithi138.stdout:35: done (15 left) 2023-06-27T12:38:07.351 INFO:tasks.rados.rados.0.smithi138.stdout:48: write initial oid 48 2023-06-27T12:38:07.351 INFO:tasks.rados.rados.0.smithi138.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2023-06-27T12:38:07.359 INFO:tasks.rados.rados.0.smithi138.stdout:48: writing smithi138138488-48 from 789929 to 1424750 tid 1 2023-06-27T12:38:07.367 INFO:tasks.rados.rados.0.smithi138.stdout:48: writing smithi138138488-48 from 2050536 to 2789929 tid 2 2023-06-27T12:38:07.368 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.368 INFO:tasks.rados.rados.0.smithi138.stdout:21: finishing write tid 3 to smithi138138488-21 2023-06-27T12:38:07.368 INFO:tasks.rados.rados.0.smithi138.stdout:21: finishing write tid 4 to smithi138138488-21 2023-06-27T12:38:07.368 INFO:tasks.rados.rados.0.smithi138.stdout:21: oid 21 updating version 0 to 13 2023-06-27T12:38:07.368 INFO:tasks.rados.rados.0.smithi138.stdout:21: oid 21 updating version 13 to 15 2023-06-27T12:38:07.368 INFO:tasks.rados.rados.0.smithi138.stdout:21: oid 21 version 15 is already newer than 15 2023-06-27T12:38:07.368 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 21 v 15 (ObjNum 20 snap 0 seq_num 20) dirty exists 2023-06-27T12:38:07.369 INFO:tasks.rados.rados.0.smithi138.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2023-06-27T12:38:07.369 INFO:tasks.rados.rados.0.smithi138.stdout:39: finishing write tid 2 to smithi138138488-39 2023-06-27T12:38:07.369 INFO:tasks.rados.rados.0.smithi138.stdout:39: finishing write tid 3 to smithi138138488-39 2023-06-27T12:38:07.369 INFO:tasks.rados.rados.0.smithi138.stdout:39: finishing write tid 4 to smithi138138488-39 2023-06-27T12:38:07.369 INFO:tasks.rados.rados.0.smithi138.stdout:39: oid 39 updating version 0 to 12 2023-06-27T12:38:07.369 INFO:tasks.rados.rados.0.smithi138.stdout:39: oid 39 version 12 is already newer than 10 2023-06-27T12:38:07.369 INFO:tasks.rados.rados.0.smithi138.stdout:39: oid 39 version 12 is already newer than 12 2023-06-27T12:38:07.370 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 39 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2023-06-27T12:38:07.370 INFO:tasks.rados.rados.0.smithi138.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2023-06-27T12:38:07.370 INFO:tasks.rados.rados.0.smithi138.stdout:31: finishing write tid 2 to smithi138138488-31 2023-06-27T12:38:07.370 INFO:tasks.rados.rados.0.smithi138.stdout:21: done (15 left) 2023-06-27T12:38:07.370 INFO:tasks.rados.rados.0.smithi138.stdout:39: done (14 left) 2023-06-27T12:38:07.370 INFO:tasks.rados.rados.0.smithi138.stdout:49: write initial oid 49 2023-06-27T12:38:07.370 INFO:tasks.rados.rados.0.smithi138.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2023-06-27T12:38:07.376 INFO:tasks.rados.rados.0.smithi138.stdout:49: writing smithi138138488-49 from 406736 to 1116806 tid 1 2023-06-27T12:38:07.381 INFO:tasks.rados.rados.0.smithi138.stdout:49: writing smithi138138488-49 from 1709018 to 2364368 tid 2 2023-06-27T12:38:07.381 INFO:tasks.rados.rados.0.smithi138.stdout:50: write initial oid 50 2023-06-27T12:38:07.382 INFO:tasks.rados.rados.0.smithi138.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2023-06-27T12:38:07.389 INFO:tasks.rados.rados.0.smithi138.stdout:50: writing smithi138138488-50 from 423543 to 1208862 tid 1 2023-06-27T12:38:07.397 INFO:tasks.rados.rados.0.smithi138.stdout:50: writing smithi138138488-50 from 1767500 to 2566508 tid 2 2023-06-27T12:38:07.397 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.397 INFO:tasks.rados.rados.0.smithi138.stdout:31: finishing write tid 3 to smithi138138488-31 2023-06-27T12:38:07.398 INFO:tasks.rados.rados.0.smithi138.stdout:31: finishing write tid 4 to smithi138138488-31 2023-06-27T12:38:07.398 INFO:tasks.rados.rados.0.smithi138.stdout:31: oid 31 updating version 0 to 9 2023-06-27T12:38:07.398 INFO:tasks.rados.rados.0.smithi138.stdout:31: oid 31 version 9 is already newer than 9 2023-06-27T12:38:07.398 INFO:tasks.rados.rados.0.smithi138.stdout:31: oid 31 version 9 is already newer than 7 2023-06-27T12:38:07.398 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 9 (ObjNum 30 snap 0 seq_num 30) dirty exists 2023-06-27T12:38:07.398 INFO:tasks.rados.rados.0.smithi138.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2023-06-27T12:38:07.398 INFO:tasks.rados.rados.0.smithi138.stdout:40: finishing write tid 1 to smithi138138488-40 2023-06-27T12:38:07.398 INFO:tasks.rados.rados.0.smithi138.stdout:47: finishing write tid 1 to smithi138138488-47 2023-06-27T12:38:07.399 INFO:tasks.rados.rados.0.smithi138.stdout:31: done (15 left) 2023-06-27T12:38:07.399 INFO:tasks.rados.rados.0.smithi138.stdout:51: write (excl) oid 2 current snap is 0 2023-06-27T12:38:07.399 INFO:tasks.rados.rados.0.smithi138.stdout:51: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2023-06-27T12:38:07.402 INFO:tasks.rados.rados.0.smithi138.stdout:51: writing smithi138138488-2 from 440350 to 900918 tid 1 2023-06-27T12:38:07.407 INFO:tasks.rados.rados.0.smithi138.stdout:51: writing smithi138138488-2 from 1309629 to 1852295 tid 2 2023-06-27T12:38:07.413 INFO:tasks.rados.rados.0.smithi138.stdout:51: writing smithi138138488-2 from 2323973 to 2840350 tid 3 2023-06-27T12:38:07.413 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.413 INFO:tasks.rados.rados.0.smithi138.stdout:45: finishing write tid 1 to smithi138138488-45 2023-06-27T12:38:07.424 INFO:tasks.rados.rados.0.smithi138.stdout:41: finishing write tid 2 to smithi138138488-41 2023-06-27T12:38:07.427 INFO:tasks.rados.rados.0.smithi138.stdout:48: finishing write tid 1 to smithi138138488-48 2023-06-27T12:38:07.449 INFO:tasks.rados.rados.0.smithi138.stdout:38: finishing write tid 2 to smithi138138488-38 2023-06-27T12:38:07.453 INFO:tasks.rados.rados.0.smithi138.stdout:38: finishing write tid 3 to smithi138138488-38 2023-06-27T12:38:07.454 INFO:tasks.rados.rados.0.smithi138.stdout:38: finishing write tid 4 to smithi138138488-38 2023-06-27T12:38:07.454 INFO:tasks.rados.rados.0.smithi138.stdout:38: oid 38 updating version 0 to 3 2023-06-27T12:38:07.454 INFO:tasks.rados.rados.0.smithi138.stdout:38: oid 38 version 3 is already newer than 1 2023-06-27T12:38:07.454 INFO:tasks.rados.rados.0.smithi138.stdout:38: oid 38 version 3 is already newer than 3 2023-06-27T12:38:07.454 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 3 (ObjNum 37 snap 0 seq_num 37) dirty exists 2023-06-27T12:38:07.455 INFO:tasks.rados.rados.0.smithi138.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2023-06-27T12:38:07.455 INFO:tasks.rados.rados.0.smithi138.stdout:34: finishing write tid 1 to smithi138138488-34 2023-06-27T12:38:07.455 INFO:tasks.rados.rados.0.smithi138.stdout:38: done (15 left) 2023-06-27T12:38:07.455 INFO:tasks.rados.rados.0.smithi138.stdout:52: write (excl) oid 14 current snap is 0 2023-06-27T12:38:07.455 INFO:tasks.rados.rados.0.smithi138.stdout:52: seq_num 51 ranges {457157=535817,1768111=402677} 2023-06-27T12:38:07.459 INFO:tasks.rados.rados.0.smithi138.stdout:52: writing smithi138138488-14 from 457157 to 992974 tid 1 2023-06-27T12:38:07.467 INFO:tasks.rados.rados.0.smithi138.stdout:52: writing smithi138138488-14 from 1768111 to 2170788 tid 2 2023-06-27T12:38:07.467 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.467 INFO:tasks.rados.rados.0.smithi138.stdout:41: finishing write tid 3 to smithi138138488-41 2023-06-27T12:38:07.467 INFO:tasks.rados.rados.0.smithi138.stdout:41: finishing write tid 4 to smithi138138488-41 2023-06-27T12:38:07.467 INFO:tasks.rados.rados.0.smithi138.stdout:41: oid 41 updating version 0 to 9 2023-06-27T12:38:07.467 INFO:tasks.rados.rados.0.smithi138.stdout:41: oid 41 version 9 is already newer than 7 2023-06-27T12:38:07.468 INFO:tasks.rados.rados.0.smithi138.stdout:41: oid 41 version 9 is already newer than 9 2023-06-27T12:38:07.468 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 41 v 9 (ObjNum 40 snap 0 seq_num 40) dirty exists 2023-06-27T12:38:07.468 INFO:tasks.rados.rados.0.smithi138.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2023-06-27T12:38:07.468 INFO:tasks.rados.rados.0.smithi138.stdout:40: finishing write tid 2 to smithi138138488-40 2023-06-27T12:38:07.468 INFO:tasks.rados.rados.0.smithi138.stdout:41: done (15 left) 2023-06-27T12:38:07.468 INFO:tasks.rados.rados.0.smithi138.stdout:53: read oid 20 snap -1 2023-06-27T12:38:07.468 INFO:tasks.rados.rados.0.smithi138.stdout:53: expect (ObjNum 19 snap 0 seq_num 19) 2023-06-27T12:38:07.469 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:38:07.469 INFO:tasks.rados.rados.0.smithi138.stdout:50: finishing write tid 1 to smithi138138488-50 2023-06-27T12:38:07.469 INFO:tasks.rados.rados.0.smithi138.stdout:36: finishing write tid 2 to smithi138138488-36 2023-06-27T12:38:07.479 INFO:tasks.rados.rados.0.smithi138.stdout:47: finishing write tid 2 to smithi138138488-47 2023-06-27T12:38:07.481 INFO:tasks.rados.rados.0.smithi138.stdout:40: finishing write tid 3 to smithi138138488-40 2023-06-27T12:38:07.481 INFO:tasks.rados.rados.0.smithi138.stdout:45: finishing write tid 2 to smithi138138488-45 2023-06-27T12:38:07.488 INFO:tasks.rados.rados.0.smithi138.stdout:40: finishing write tid 4 to smithi138138488-40 2023-06-27T12:38:07.488 INFO:tasks.rados.rados.0.smithi138.stdout:40: oid 40 updating version 0 to 6 2023-06-27T12:38:07.488 INFO:tasks.rados.rados.0.smithi138.stdout:40: oid 40 version 6 is already newer than 6 2023-06-27T12:38:07.488 INFO:tasks.rados.rados.0.smithi138.stdout:40: oid 40 version 6 is already newer than 4 2023-06-27T12:38:07.488 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2023-06-27T12:38:07.488 INFO:tasks.rados.rados.0.smithi138.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2023-06-27T12:38:07.488 INFO:tasks.rados.rados.0.smithi138.stdout:40: done (15 left) 2023-06-27T12:38:07.489 INFO:tasks.rados.rados.0.smithi138.stdout:54: delete oid 23 current snap is 0 2023-06-27T12:38:07.489 INFO:tasks.rados.rados.0.smithi138.stdout:46: finishing write tid 2 to smithi138138488-46 2023-06-27T12:38:07.509 INFO:tasks.rados.rados.0.smithi138.stdout:36: finishing write tid 3 to smithi138138488-36 2023-06-27T12:38:07.509 INFO:tasks.rados.rados.0.smithi138.stdout:36: finishing write tid 4 to smithi138138488-36 2023-06-27T12:38:07.510 INFO:tasks.rados.rados.0.smithi138.stdout:36: oid 36 updating version 0 to 12 2023-06-27T12:38:07.510 INFO:tasks.rados.rados.0.smithi138.stdout:36: oid 36 version 12 is already newer than 10 2023-06-27T12:38:07.510 INFO:tasks.rados.rados.0.smithi138.stdout:36: oid 36 version 12 is already newer than 12 2023-06-27T12:38:07.510 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 12 (ObjNum 35 snap 0 seq_num 35) dirty exists 2023-06-27T12:38:07.510 INFO:tasks.rados.rados.0.smithi138.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2023-06-27T12:38:07.518 INFO:tasks.rados.rados.0.smithi138.stdout:47: finishing write tid 3 to smithi138138488-47 2023-06-27T12:38:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:46: finishing write tid 3 to smithi138138488-46 2023-06-27T12:38:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:47: finishing write tid 4 to smithi138138488-47 2023-06-27T12:38:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:47: oid 47 updating version 0 to 10 2023-06-27T12:38:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:47: oid 47 updating version 10 to 12 2023-06-27T12:38:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:47: oid 47 version 12 is already newer than 12 2023-06-27T12:38:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 47 v 12 (ObjNum 46 snap 0 seq_num 46) dirty exists 2023-06-27T12:38:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2023-06-27T12:38:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:46: finishing write tid 4 to smithi138138488-46 2023-06-27T12:38:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:46: oid 46 updating version 0 to 7 2023-06-27T12:38:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:46: oid 46 updating version 7 to 9 2023-06-27T12:38:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:46: oid 46 version 9 is already newer than 9 2023-06-27T12:38:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 9 (ObjNum 45 snap 0 seq_num 45) dirty exists 2023-06-27T12:38:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2023-06-27T12:38:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:48: finishing write tid 2 to smithi138138488-48 2023-06-27T12:38:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:52: finishing write tid 1 to smithi138138488-14 2023-06-27T12:38:07.522 INFO:tasks.rados.rados.0.smithi138.stdout:45: finishing write tid 3 to smithi138138488-45 2023-06-27T12:38:07.522 INFO:tasks.rados.rados.0.smithi138.stdout:45: finishing write tid 4 to smithi138138488-45 2023-06-27T12:38:07.523 INFO:tasks.rados.rados.0.smithi138.stdout:45: oid 45 updating version 0 to 15 2023-06-27T12:38:07.523 INFO:tasks.rados.rados.0.smithi138.stdout:45: oid 45 version 15 is already newer than 13 2023-06-27T12:38:07.523 INFO:tasks.rados.rados.0.smithi138.stdout:45: oid 45 version 15 is already newer than 15 2023-06-27T12:38:07.523 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 15 (ObjNum 44 snap 0 seq_num 44) dirty exists 2023-06-27T12:38:07.523 INFO:tasks.rados.rados.0.smithi138.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2023-06-27T12:38:07.529 INFO:tasks.rados.rados.0.smithi138.stdout:48: finishing write tid 3 to smithi138138488-48 2023-06-27T12:38:07.536 INFO:tasks.rados.rados.0.smithi138.stdout:37: finishing write tid 1 to smithi138138488-37 2023-06-27T12:38:07.539 INFO:tasks.rados.rados.0.smithi138.stdout:48: finishing write tid 4 to smithi138138488-48 2023-06-27T12:38:07.539 INFO:tasks.rados.rados.0.smithi138.stdout:48: oid 48 updating version 0 to 4 2023-06-27T12:38:07.539 INFO:tasks.rados.rados.0.smithi138.stdout:48: oid 48 updating version 4 to 6 2023-06-27T12:38:07.539 INFO:tasks.rados.rados.0.smithi138.stdout:48: oid 48 version 6 is already newer than 6 2023-06-27T12:38:07.539 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 6 (ObjNum 47 snap 0 seq_num 47) dirty exists 2023-06-27T12:38:07.539 INFO:tasks.rados.rados.0.smithi138.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2023-06-27T12:38:07.540 INFO:tasks.rados.rados.0.smithi138.stdout:34: finishing write tid 2 to smithi138138488-34 2023-06-27T12:38:07.553 INFO:tasks.rados.rados.0.smithi138.stdout:52: finishing write tid 2 to smithi138138488-14 2023-06-27T12:38:07.555 INFO:tasks.rados.rados.0.smithi138.stdout:52: finishing write tid 3 to smithi138138488-14 2023-06-27T12:38:07.556 INFO:tasks.rados.rados.0.smithi138.stdout:52: finishing write tid 4 to smithi138138488-14 2023-06-27T12:38:07.556 INFO:tasks.rados.rados.0.smithi138.stdout:52: oid 14 updating version 0 to 18 2023-06-27T12:38:07.556 INFO:tasks.rados.rados.0.smithi138.stdout:52: oid 14 version 18 is already newer than 18 2023-06-27T12:38:07.557 INFO:tasks.rados.rados.0.smithi138.stdout:52: oid 14 version 18 is already newer than 16 2023-06-27T12:38:07.557 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 18 (ObjNum 51 snap 0 seq_num 51) dirty exists 2023-06-27T12:38:07.557 INFO:tasks.rados.rados.0.smithi138.stdout:52: left oid 14 (ObjNum 51 snap 0 seq_num 51) 2023-06-27T12:38:07.565 INFO:tasks.rados.rados.0.smithi138.stdout:53: expect (ObjNum 19 snap 0 seq_num 19) 2023-06-27T12:38:07.665 INFO:tasks.rados.rados.0.smithi138.stdout:37: finishing write tid 2 to smithi138138488-37 2023-06-27T12:38:07.666 INFO:tasks.rados.rados.0.smithi138.stdout:50: finishing write tid 2 to smithi138138488-50 2023-06-27T12:38:07.666 INFO:tasks.rados.rados.0.smithi138.stdout:37: finishing write tid 3 to smithi138138488-37 2023-06-27T12:38:07.666 INFO:tasks.rados.rados.0.smithi138.stdout:37: finishing write tid 4 to smithi138138488-37 2023-06-27T12:38:07.666 INFO:tasks.rados.rados.0.smithi138.stdout:37: oid 37 updating version 0 to 15 2023-06-27T12:38:07.667 INFO:tasks.rados.rados.0.smithi138.stdout:37: oid 37 version 15 is already newer than 13 2023-06-27T12:38:07.667 INFO:tasks.rados.rados.0.smithi138.stdout:37: oid 37 version 15 is already newer than 15 2023-06-27T12:38:07.667 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 15 (ObjNum 36 snap 0 seq_num 36) dirty exists 2023-06-27T12:38:07.667 INFO:tasks.rados.rados.0.smithi138.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2023-06-27T12:38:07.667 INFO:tasks.rados.rados.0.smithi138.stdout:36: done (15 left) 2023-06-27T12:38:07.668 INFO:tasks.rados.rados.0.smithi138.stdout:37: done (14 left) 2023-06-27T12:38:07.668 INFO:tasks.rados.rados.0.smithi138.stdout:45: done (13 left) 2023-06-27T12:38:07.668 INFO:tasks.rados.rados.0.smithi138.stdout:46: done (12 left) 2023-06-27T12:38:07.668 INFO:tasks.rados.rados.0.smithi138.stdout:47: done (11 left) 2023-06-27T12:38:07.668 INFO:tasks.rados.rados.0.smithi138.stdout:48: done (10 left) 2023-06-27T12:38:07.668 INFO:tasks.rados.rados.0.smithi138.stdout:52: done (9 left) 2023-06-27T12:38:07.668 INFO:tasks.rados.rados.0.smithi138.stdout:53: done (8 left) 2023-06-27T12:38:07.669 INFO:tasks.rados.rados.0.smithi138.stdout:54: done (7 left) 2023-06-27T12:38:07.669 INFO:tasks.rados.rados.0.smithi138.stdout:55: write oid 7 current snap is 0 2023-06-27T12:38:07.669 INFO:tasks.rados.rados.0.smithi138.stdout:50: finishing write tid 3 to smithi138138488-50 2023-06-27T12:38:07.669 INFO:tasks.rados.rados.0.smithi138.stdout:50: finishing write tid 4 to smithi138138488-50 2023-06-27T12:38:07.669 INFO:tasks.rados.rados.0.smithi138.stdout:50: oid 50 updating version 0 to 3 2023-06-27T12:38:07.669 INFO:tasks.rados.rados.0.smithi138.stdout:50: oid 50 version 3 is already newer than 3 2023-06-27T12:38:07.669 INFO:tasks.rados.rados.0.smithi138.stdout:50: oid 50 version 3 is already newer than 1 2023-06-27T12:38:07.669 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 3 (ObjNum 49 snap 0 seq_num 49) dirty exists 2023-06-27T12:38:07.670 INFO:tasks.rados.rados.0.smithi138.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2023-06-27T12:38:07.670 INFO:tasks.rados.rados.0.smithi138.stdout:43: finishing write tid 1 to smithi138138488-43 2023-06-27T12:38:07.670 INFO:tasks.rados.rados.0.smithi138.stdout:55: seq_num 53 ranges {490771=686315,1885075=406346} 2023-06-27T12:38:07.672 INFO:tasks.rados.rados.0.smithi138.stdout:55: writing smithi138138488-7 from 490771 to 1177086 tid 1 2023-06-27T12:38:07.679 INFO:tasks.rados.rados.0.smithi138.stdout:55: writing smithi138138488-7 from 1885075 to 2291421 tid 2 2023-06-27T12:38:07.679 INFO:tasks.rados.rados.0.smithi138.stdout:50: done (7 left) 2023-06-27T12:38:07.679 INFO:tasks.rados.rados.0.smithi138.stdout:56: write oid 37 current snap is 0 2023-06-27T12:38:07.679 INFO:tasks.rados.rados.0.smithi138.stdout:34: finishing write tid 3 to smithi138138488-34 2023-06-27T12:38:07.680 INFO:tasks.rados.rados.0.smithi138.stdout:34: finishing write tid 4 to smithi138138488-34 2023-06-27T12:38:07.680 INFO:tasks.rados.rados.0.smithi138.stdout:34: oid 34 updating version 0 to 12 2023-06-27T12:38:07.680 INFO:tasks.rados.rados.0.smithi138.stdout:34: oid 34 version 12 is already newer than 10 2023-06-27T12:38:07.680 INFO:tasks.rados.rados.0.smithi138.stdout:34: oid 34 version 12 is already newer than 12 2023-06-27T12:38:07.680 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 12 (ObjNum 33 snap 0 seq_num 33) dirty exists 2023-06-27T12:38:07.680 INFO:tasks.rados.rados.0.smithi138.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2023-06-27T12:38:07.680 INFO:tasks.rados.rados.0.smithi138.stdout:42: finishing write tid 1 to smithi138138488-42 2023-06-27T12:38:07.681 INFO:tasks.rados.rados.0.smithi138.stdout:49: finishing write tid 1 to smithi138138488-49 2023-06-27T12:38:07.681 INFO:tasks.rados.rados.0.smithi138.stdout:51: finishing write tid 1 to smithi138138488-2 2023-06-27T12:38:07.681 INFO:tasks.rados.rados.0.smithi138.stdout:56: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2023-06-27T12:38:07.685 INFO:tasks.rados.rados.0.smithi138.stdout:56: writing smithi138138488-37 from 507578 to 985495 tid 1 2023-06-27T12:38:07.691 INFO:tasks.rados.rados.0.smithi138.stdout:56: writing smithi138138488-37 from 1543557 to 2093561 tid 2 2023-06-27T12:38:07.696 INFO:tasks.rados.rados.0.smithi138.stdout:56: writing smithi138138488-37 from 2833665 to 2907578 tid 3 2023-06-27T12:38:07.697 INFO:tasks.rados.rados.0.smithi138.stdout:34: done (7 left) 2023-06-27T12:38:07.697 INFO:tasks.rados.rados.0.smithi138.stdout:57: write (excl) oid 18 current snap is 0 2023-06-27T12:38:07.697 INFO:tasks.rados.rados.0.smithi138.stdout:43: finishing write tid 2 to smithi138138488-43 2023-06-27T12:38:07.697 INFO:tasks.rados.rados.0.smithi138.stdout:43: finishing write tid 3 to smithi138138488-43 2023-06-27T12:38:07.697 INFO:tasks.rados.rados.0.smithi138.stdout:43: finishing write tid 4 to smithi138138488-43 2023-06-27T12:38:07.697 INFO:tasks.rados.rados.0.smithi138.stdout:43: oid 43 updating version 0 to 10 2023-06-27T12:38:07.697 INFO:tasks.rados.rados.0.smithi138.stdout:43: oid 43 updating version 10 to 12 2023-06-27T12:38:07.697 INFO:tasks.rados.rados.0.smithi138.stdout:43: oid 43 version 12 is already newer than 12 2023-06-27T12:38:07.698 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 12 (ObjNum 42 snap 0 seq_num 42) dirty exists 2023-06-27T12:38:07.698 INFO:tasks.rados.rados.0.smithi138.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2023-06-27T12:38:07.698 INFO:tasks.rados.rados.0.smithi138.stdout:57: seq_num 55 ranges {524385=553166,1602039=410015,2577441=346944} 2023-06-27T12:38:07.702 INFO:tasks.rados.rados.0.smithi138.stdout:57: writing smithi138138488-18 from 524385 to 1077551 tid 1 2023-06-27T12:38:07.707 INFO:tasks.rados.rados.0.smithi138.stdout:57: writing smithi138138488-18 from 1602039 to 2012054 tid 2 2023-06-27T12:38:07.713 INFO:tasks.rados.rados.0.smithi138.stdout:57: writing smithi138138488-18 from 2577441 to 2924385 tid 3 2023-06-27T12:38:07.713 INFO:tasks.rados.rados.0.smithi138.stdout:43: done (7 left) 2023-06-27T12:38:07.713 INFO:tasks.rados.rados.0.smithi138.stdout:58: read oid 39 snap -1 2023-06-27T12:38:07.713 INFO:tasks.rados.rados.0.smithi138.stdout:58: expect (ObjNum 38 snap 0 seq_num 38) 2023-06-27T12:38:07.713 INFO:tasks.rados.rados.0.smithi138.stdout:59: write (excl) oid 35 current snap is 0 2023-06-27T12:38:07.713 INFO:tasks.rados.rados.0.smithi138.stdout:44: finishing write tid 1 to smithi138138488-44 2023-06-27T12:38:07.714 INFO:tasks.rados.rados.0.smithi138.stdout:59: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2023-06-27T12:38:07.719 INFO:tasks.rados.rados.0.smithi138.stdout:59: writing smithi138138488-35 from 541192 to 1169607 tid 1 2023-06-27T12:38:07.727 INFO:tasks.rados.rados.0.smithi138.stdout:59: writing smithi138138488-35 from 1660521 to 2214194 tid 2 2023-06-27T12:38:07.732 INFO:tasks.rados.rados.0.smithi138.stdout:59: writing smithi138138488-35 from 2888511 to 2941192 tid 3 2023-06-27T12:38:07.732 INFO:tasks.rados.rados.0.smithi138.stdout:60: delete oid 40 current snap is 0 2023-06-27T12:38:07.732 INFO:tasks.rados.rados.0.smithi138.stdout:42: finishing write tid 2 to smithi138138488-42 2023-06-27T12:38:07.732 INFO:tasks.rados.rados.0.smithi138.stdout:51: finishing write tid 2 to smithi138138488-2 2023-06-27T12:38:07.732 INFO:tasks.rados.rados.0.smithi138.stdout:51: finishing write tid 3 to smithi138138488-2 2023-06-27T12:38:07.732 INFO:tasks.rados.rados.0.smithi138.stdout:51: finishing write tid 4 to smithi138138488-2 2023-06-27T12:38:07.733 INFO:tasks.rados.rados.0.smithi138.stdout:51: finishing write tid 5 to smithi138138488-2 2023-06-27T12:38:07.733 INFO:tasks.rados.rados.0.smithi138.stdout:51: oid 2 updating version 0 to 19 2023-06-27T12:38:07.733 INFO:tasks.rados.rados.0.smithi138.stdout:51: oid 2 version 19 is already newer than 19 2023-06-27T12:38:07.733 INFO:tasks.rados.rados.0.smithi138.stdout:51: oid 2 version 19 is already newer than 19 2023-06-27T12:38:07.733 INFO:tasks.rados.rados.0.smithi138.stdout:51: oid 2 version 19 is already newer than 16 2023-06-27T12:38:07.733 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 19 (ObjNum 50 snap 0 seq_num 50) dirty exists 2023-06-27T12:38:07.733 INFO:tasks.rados.rados.0.smithi138.stdout:51: left oid 2 (ObjNum 50 snap 0 seq_num 50) 2023-06-27T12:38:07.733 INFO:tasks.rados.rados.0.smithi138.stdout:42: finishing write tid 3 to smithi138138488-42 2023-06-27T12:38:07.734 INFO:tasks.rados.rados.0.smithi138.stdout:49: finishing write tid 2 to smithi138138488-49 2023-06-27T12:38:07.734 INFO:tasks.rados.rados.0.smithi138.stdout:42: finishing write tid 4 to smithi138138488-42 2023-06-27T12:38:07.734 INFO:tasks.rados.rados.0.smithi138.stdout:42: oid 42 updating version 0 to 7 2023-06-27T12:38:07.734 INFO:tasks.rados.rados.0.smithi138.stdout:42: oid 42 updating version 7 to 9 2023-06-27T12:38:07.734 INFO:tasks.rados.rados.0.smithi138.stdout:42: oid 42 version 9 is already newer than 9 2023-06-27T12:38:07.734 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 9 (ObjNum 41 snap 0 seq_num 41) dirty exists 2023-06-27T12:38:07.734 INFO:tasks.rados.rados.0.smithi138.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2023-06-27T12:38:07.734 INFO:tasks.rados.rados.0.smithi138.stdout:49: finishing write tid 3 to smithi138138488-49 2023-06-27T12:38:07.735 INFO:tasks.rados.rados.0.smithi138.stdout:49: finishing write tid 4 to smithi138138488-49 2023-06-27T12:38:07.735 INFO:tasks.rados.rados.0.smithi138.stdout:49: oid 49 updating version 0 to 7 2023-06-27T12:38:07.735 INFO:tasks.rados.rados.0.smithi138.stdout:49: oid 49 updating version 7 to 9 2023-06-27T12:38:07.735 INFO:tasks.rados.rados.0.smithi138.stdout:49: oid 49 version 9 is already newer than 9 2023-06-27T12:38:07.735 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2023-06-27T12:38:07.735 INFO:tasks.rados.rados.0.smithi138.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2023-06-27T12:38:07.735 INFO:tasks.rados.rados.0.smithi138.stdout:44: finishing write tid 2 to smithi138138488-44 2023-06-27T12:38:07.735 INFO:tasks.rados.rados.0.smithi138.stdout:44: finishing write tid 3 to smithi138138488-44 2023-06-27T12:38:07.736 INFO:tasks.rados.rados.0.smithi138.stdout:44: finishing write tid 4 to smithi138138488-44 2023-06-27T12:38:07.736 INFO:tasks.rados.rados.0.smithi138.stdout:44: oid 44 updating version 0 to 13 2023-06-27T12:38:07.736 INFO:tasks.rados.rados.0.smithi138.stdout:44: oid 44 updating version 13 to 15 2023-06-27T12:38:07.736 INFO:tasks.rados.rados.0.smithi138.stdout:44: oid 44 version 15 is already newer than 15 2023-06-27T12:38:07.736 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 15 (ObjNum 43 snap 0 seq_num 43) dirty exists 2023-06-27T12:38:07.736 INFO:tasks.rados.rados.0.smithi138.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2023-06-27T12:38:07.736 INFO:tasks.rados.rados.0.smithi138.stdout:55: finishing write tid 1 to smithi138138488-7 2023-06-27T12:38:07.736 INFO:tasks.rados.rados.0.smithi138.stdout:55: finishing write tid 2 to smithi138138488-7 2023-06-27T12:38:07.744 INFO:tasks.rados.rados.0.smithi138.stdout:55: finishing write tid 3 to smithi138138488-7 2023-06-27T12:38:07.746 INFO:tasks.rados.rados.0.smithi138.stdout:56: finishing write tid 1 to smithi138138488-37 2023-06-27T12:38:07.762 INFO:tasks.rados.rados.0.smithi138.stdout:56: finishing write tid 2 to smithi138138488-37 2023-06-27T12:38:07.764 INFO:tasks.rados.rados.0.smithi138.stdout:59: finishing write tid 1 to smithi138138488-35 2023-06-27T12:38:07.766 INFO:tasks.rados.rados.0.smithi138.stdout:56: finishing write tid 3 to smithi138138488-37 2023-06-27T12:38:07.768 INFO:tasks.rados.rados.0.smithi138.stdout:56: finishing write tid 4 to smithi138138488-37 2023-06-27T12:38:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:56: finishing write tid 5 to smithi138138488-37 2023-06-27T12:38:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:56: oid 37 updating version 0 to 19 2023-06-27T12:38:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:56: oid 37 version 19 is already newer than 19 2023-06-27T12:38:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:56: oid 37 version 19 is already newer than 19 2023-06-27T12:38:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:56: oid 37 version 19 is already newer than 18 2023-06-27T12:38:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 19 (ObjNum 54 snap 0 seq_num 54) dirty exists 2023-06-27T12:38:07.773 INFO:tasks.rados.rados.0.smithi138.stdout:56: left oid 37 (ObjNum 54 snap 0 seq_num 54) 2023-06-27T12:38:07.784 INFO:tasks.rados.rados.0.smithi138.stdout:55: finishing write tid 4 to smithi138138488-7 2023-06-27T12:38:07.785 INFO:tasks.rados.rados.0.smithi138.stdout:55: oid 7 updating version 0 to 12 2023-06-27T12:38:07.785 INFO:tasks.rados.rados.0.smithi138.stdout:55: oid 7 version 12 is already newer than 12 2023-06-27T12:38:07.785 INFO:tasks.rados.rados.0.smithi138.stdout:55: oid 7 version 12 is already newer than 12 2023-06-27T12:38:07.785 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 12 (ObjNum 53 snap 0 seq_num 53) dirty exists 2023-06-27T12:38:07.785 INFO:tasks.rados.rados.0.smithi138.stdout:55: left oid 7 (ObjNum 53 snap 0 seq_num 53) 2023-06-27T12:38:07.788 INFO:tasks.rados.rados.0.smithi138.stdout:42: done (9 left) 2023-06-27T12:38:07.788 INFO:tasks.rados.rados.0.smithi138.stdout:44: done (8 left) 2023-06-27T12:38:07.788 INFO:tasks.rados.rados.0.smithi138.stdout:49: done (7 left) 2023-06-27T12:38:07.789 INFO:tasks.rados.rados.0.smithi138.stdout:51: done (6 left) 2023-06-27T12:38:07.789 INFO:tasks.rados.rados.0.smithi138.stdout:55: done (5 left) 2023-06-27T12:38:07.789 INFO:tasks.rados.rados.0.smithi138.stdout:56: done (4 left) 2023-06-27T12:38:07.789 INFO:tasks.rados.rados.0.smithi138.stdout:60: done (3 left) 2023-06-27T12:38:07.789 INFO:tasks.rados.rados.0.smithi138.stdout:61: delete oid 46 current snap is 0 2023-06-27T12:38:07.800 INFO:tasks.rados.rados.0.smithi138.stdout:58: expect (ObjNum 38 snap 0 seq_num 38) 2023-06-27T12:38:07.904 INFO:tasks.rados.rados.0.smithi138.stdout:59: finishing write tid 2 to smithi138138488-35 2023-06-27T12:38:07.905 INFO:tasks.rados.rados.0.smithi138.stdout:59: finishing write tid 3 to smithi138138488-35 2023-06-27T12:38:07.905 INFO:tasks.rados.rados.0.smithi138.stdout:58: done (3 left) 2023-06-27T12:38:07.905 INFO:tasks.rados.rados.0.smithi138.stdout:61: done (2 left) 2023-06-27T12:38:07.905 INFO:tasks.rados.rados.0.smithi138.stdout:62: delete oid 38 current snap is 0 2023-06-27T12:38:07.905 INFO:tasks.rados.rados.0.smithi138.stdout:59: finishing write tid 4 to smithi138138488-35 2023-06-27T12:38:07.905 INFO:tasks.rados.rados.0.smithi138.stdout:59: finishing write tid 5 to smithi138138488-35 2023-06-27T12:38:07.905 INFO:tasks.rados.rados.0.smithi138.stdout:59: oid 35 updating version 0 to 20 2023-06-27T12:38:07.906 INFO:tasks.rados.rados.0.smithi138.stdout:59: oid 35 updating version 20 to 23 2023-06-27T12:38:07.906 INFO:tasks.rados.rados.0.smithi138.stdout:59: oid 35 version 23 is already newer than 23 2023-06-27T12:38:07.906 INFO:tasks.rados.rados.0.smithi138.stdout:59: oid 35 version 23 is already newer than 23 2023-06-27T12:38:07.906 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 23 (ObjNum 56 snap 0 seq_num 56) dirty exists 2023-06-27T12:38:07.906 INFO:tasks.rados.rados.0.smithi138.stdout:59: left oid 35 (ObjNum 56 snap 0 seq_num 56) 2023-06-27T12:38:07.906 INFO:tasks.rados.rados.0.smithi138.stdout:57: finishing write tid 1 to smithi138138488-18 2023-06-27T12:38:07.906 INFO:tasks.rados.rados.0.smithi138.stdout:57: finishing write tid 2 to smithi138138488-18 2023-06-27T12:38:07.907 INFO:tasks.rados.rados.0.smithi138.stdout:57: finishing write tid 3 to smithi138138488-18 2023-06-27T12:38:07.907 INFO:tasks.rados.rados.0.smithi138.stdout:57: finishing write tid 4 to smithi138138488-18 2023-06-27T12:38:07.907 INFO:tasks.rados.rados.0.smithi138.stdout:57: finishing write tid 5 to smithi138138488-18 2023-06-27T12:38:07.907 INFO:tasks.rados.rados.0.smithi138.stdout:57: oid 18 updating version 0 to 23 2023-06-27T12:38:07.907 INFO:tasks.rados.rados.0.smithi138.stdout:57: oid 18 version 23 is already newer than 23 2023-06-27T12:38:07.907 INFO:tasks.rados.rados.0.smithi138.stdout:57: oid 18 version 23 is already newer than 20 2023-06-27T12:38:07.907 INFO:tasks.rados.rados.0.smithi138.stdout:57: oid 18 version 23 is already newer than 23 2023-06-27T12:38:07.907 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 23 (ObjNum 55 snap 0 seq_num 55) dirty exists 2023-06-27T12:38:07.908 INFO:tasks.rados.rados.0.smithi138.stdout:57: left oid 18 (ObjNum 55 snap 0 seq_num 55) 2023-06-27T12:38:07.949 INFO:tasks.rados.rados.0.smithi138.stdout:57: done (2 left) 2023-06-27T12:38:07.950 INFO:tasks.rados.rados.0.smithi138.stdout:59: done (1 left) 2023-06-27T12:38:07.950 INFO:tasks.rados.rados.0.smithi138.stdout:62: done (0 left) 2023-06-27T12:38:07.950 INFO:tasks.rados.rados.0.smithi138.stdout:63: read oid 15 snap -1 2023-06-27T12:38:07.950 INFO:tasks.rados.rados.0.smithi138.stdout:63: expect (ObjNum 14 snap 0 seq_num 14) 2023-06-27T12:38:07.950 INFO:tasks.rados.rados.0.smithi138.stdout:64: snap_create 2023-06-27T12:38:07.960 INFO:tasks.rados.rados.0.smithi138.stdout:63: expect (ObjNum 14 snap 0 seq_num 14) 2023-06-27T12:38:08.065 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:08.065 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:08.066 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:08.193 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:08.186+0000 7fa940741700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:08.196 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:08.192+0000 7fa940741700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:08.390 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:08.397 INFO:tasks.rados.rados.0.smithi138.stdout:63: done (1 left) 2023-06-27T12:38:08.397 INFO:tasks.rados.rados.0.smithi138.stdout:64: done (0 left) 2023-06-27T12:38:08.398 INFO:tasks.rados.rados.0.smithi138.stdout:65: write oid 40 current snap is 1 2023-06-27T12:38:08.398 INFO:tasks.rados.rados.0.smithi138.stdout:65: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2023-06-27T12:38:08.401 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-06-27T12:38:07.906998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":107748,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379612,"statfs":{"total":386547056640,"available":386436722688,"internally_reserved":0,"allocated":462848,"data_stored":1877220,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.539577"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.399531+0000","last_change":"2023-06-27T12:38:03.399531+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.399531+0000","last_undegraded":"2023-06-27T12:38:03.399531+0000","last_fullsized":"2023-06-27T12:38:03.399531+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.431762+0000","last_change":"2023-06-27T12:38:03.431762+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.431762+0000","last_undegraded":"2023-06-27T12:38:03.431762+0000","last_fullsized":"2023-06-27T12:38:03.431762+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.426023+0000","last_change":"2023-06-27T12:38:03.426023+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.426023+0000","last_undegraded":"2023-06-27T12:38:03.426023+0000","last_fullsized":"2023-06-27T12:38:03.426023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.436262+0000","last_change":"2023-06-27T12:38:03.436262+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.436262+0000","last_undegraded":"2023-06-27T12:38:03.436262+0000","last_fullsized":"2023-06-27T12:38:03.436262+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.410536+0000","last_change":"2023-06-27T12:38:03.410536+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.410536+0000","last_undegraded":"2023-06-27T12:38:03.410536+0000","last_fullsized":"2023-06-27T12:38:03.410536+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.421063+0000","last_change":"2023-06-27T12:38:03.421063+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.421063+0000","last_undegraded":"2023-06-27T12:38:03.421063+0000","last_fullsized":"2023-06-27T12:38:03.421063+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.411168+0000","last_change":"2023-06-27T12:38:03.411168+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.411168+0000","last_undegraded":"2023-06-27T12:38:03.411168+0000","last_fullsized":"2023-06-27T12:38:03.411168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.412299+0000","last_change":"2023-06-27T12:38:03.412299+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.412299+0000","last_undegraded":"2023-06-27T12:38:03.412299+0000","last_fullsized":"2023-06-27T12:38:03.412299+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-06-27T12:38:03.392467+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:03.392467+0000","last_peered":"2023-06-27T12:38:03.392467+0000","last_clean":"2023-06-27T12:38:03.392467+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:03.392467+0000","last_undegraded":"2023-06-27T12:38:03.392467+0000","last_fullsized":"2023-06-27T12:38:03.392467+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.405317+0000","last_change":"2023-06-27T12:38:03.405317+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.405317+0000","last_undegraded":"2023-06-27T12:38:03.405317+0000","last_fullsized":"2023-06-27T12:38:03.405317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-06-27T12:38:03.393538+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:03.393538+0000","last_peered":"2023-06-27T12:38:03.393538+0000","last_clean":"2023-06-27T12:38:03.393538+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:03.393538+0000","last_undegraded":"2023-06-27T12:38:03.393538+0000","last_fullsized":"2023-06-27T12:38:03.393538+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.401551+0000","last_change":"2023-06-27T12:38:03.401551+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.401551+0000","last_undegraded":"2023-06-27T12:38:03.401551+0000","last_fullsized":"2023-06-27T12:38:03.401551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-06-27T12:38:03.387769+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:03.387769+0000","last_peered":"2023-06-27T12:38:03.387769+0000","last_clean":"2023-06-27T12:38:03.387769+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:03.387769+0000","last_undegraded":"2023-06-27T12:38:03.387769+0000","last_fullsized":"2023-06-27T12:38:03.387769+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.420427+0000","last_change":"2023-06-27T12:38:03.420427+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.420427+0000","last_undegraded":"2023-06-27T12:38:03.420427+0000","last_fullsized":"2023-06-27T12:38:03.420427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-06-27T12:38:03.384881+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:03.384881+0000","last_peered":"2023-06-27T12:38:03.384881+0000","last_clean":"2023-06-27T12:38:03.384881+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:03.384881+0000","last_undegraded":"2023-06-27T12:38:03.384881+0000","last_fullsized":"2023-06-27T12:38:03.384881+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":142,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-06-27T12:38:03.394260+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:03.394260+0000","last_peered":"2023-06-27T12:38:03.394260+0000","last_clean":"2023-06-27T12:38:03.394260+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:03.394260+0000","last_undegraded":"2023-06-27T12:38:03.394260+0000","last_fullsized":"2023-06-27T12:38:03.394260+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.395161+0000","last_change":"2023-06-27T12:38:03.395161+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.395161+0000","last_undegraded":"2023-06-27T12:38:03.395161+0000","last_fullsized":"2023-06-27T12:38:03.395161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-06-27T12:38:03.392109+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:03.392109+0000","last_peered":"2023-06-27T12:38:03.392109+0000","last_clean":"2023-06-27T12:38:03.392109+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:03.392109+0000","last_undegraded":"2023-06-27T12:38:03.392109+0000","last_fullsized":"2023-06-27T12:38:03.392109+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.431851+0000","last_change":"2023-06-27T12:38:03.431851+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.431851+0000","last_undegraded":"2023-06-27T12:38:03.431851+0000","last_fullsized":"2023-06-27T12:38:03.431851+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-06-27T12:38:03.391889+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:03.391889+0000","last_peered":"2023-06-27T12:38:03.391889+0000","last_clean":"2023-06-27T12:38:03.391889+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:03.391889+0000","last_undegraded":"2023-06-27T12:38:03.391889+0000","last_fullsized":"2023-06-27T12:38:03.391889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.410309+0000","last_change":"2023-06-27T12:38:03.410309+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.410309+0000","last_undegraded":"2023-06-27T12:38:03.410309+0000","last_fullsized":"2023-06-27T12:38:03.410309+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.399382+0000","last_change":"2023-06-27T12:38:03.399382+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.399382+0000","last_undegraded":"2023-06-27T12:38:03.399382+0000","last_fullsized":"2023-06-27T12:38:03.399382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-06-27T12:38:03.392228+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:03.392228+0000","last_peered":"2023-06-27T12:38:03.392228+0000","last_clean":"2023-06-27T12:38:03.392228+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:03.392228+0000","last_undegraded":"2023-06-27T12:38:03.392228+0000","last_fullsized":"2023-06-27T12:38:03.392228+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-06-27T12:38:03.389523+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:03.389523+0000","last_peered":"2023-06-27T12:38:03.389523+0000","last_clean":"2023-06-27T12:38:03.389523+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:03.389523+0000","last_undegraded":"2023-06-27T12:38:03.389523+0000","last_fullsized":"2023-06-27T12:38:03.389523+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":16,"state":"creating+peering","last_fresh":"2023-06-27T12:38:03.418842+0000","last_change":"2023-06-27T12:38:03.418842+0000","last_active":"2023-06-27T12:38:03.365505+0000","last_peered":"2023-06-27T12:38:03.365505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-27T12:38:03.418842+0000","last_undegraded":"2023-06-27T12:38:03.418842+0000","last_fullsized":"2023-06-27T12:38:03.418842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":131072,"data_stored":616897,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":131072,"data_stored":616897,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705675,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26952,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344888,"statfs":{"total":96636764160,"available":96609165312,"internally_reserved":0,"allocated":131072,"data_stored":616897,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26892,"kb_used_data":68,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344948,"statfs":{"total":96636764160,"available":96609226752,"internally_reserved":0,"allocated":69632,"data_stored":26529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:08.403 INFO:tasks.rados.rados.0.smithi138.stdout:65: writing smithi138138488-40 from 608420 to 1137831 tid 1 2023-06-27T12:38:08.411 INFO:tasks.rados.rados.0.smithi138.stdout:65: writing smithi138138488-40 from 1778096 to 2339107 tid 2 2023-06-27T12:38:08.416 INFO:tasks.rados.rados.0.smithi138.stdout:65: writing smithi138138488-40 from 2881850 to 3008420 tid 3 2023-06-27T12:38:08.416 INFO:tasks.rados.rados.0.smithi138.stdout:66: read oid 48 snap 0 2023-06-27T12:38:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:66: expect (ObjNum 47 snap 0 seq_num 47) 2023-06-27T12:38:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:67: read oid 28 snap 0 2023-06-27T12:38:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:67: expect (ObjNum 27 snap 0 seq_num 27) 2023-06-27T12:38:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:68: read oid 13 snap -1 2023-06-27T12:38:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:68: expect (ObjNum 12 snap 0 seq_num 12) 2023-06-27T12:38:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:69: read oid 43 snap -1 2023-06-27T12:38:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:69: expect (ObjNum 42 snap 0 seq_num 42) 2023-06-27T12:38:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:70: read oid 36 snap -1 2023-06-27T12:38:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:70: expect (ObjNum 35 snap 0 seq_num 35) 2023-06-27T12:38:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:71: read oid 50 snap -1 2023-06-27T12:38:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:71: expect (ObjNum 49 snap 0 seq_num 49) 2023-06-27T12:38:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:72: read oid 8 snap 0 2023-06-27T12:38:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:72: expect (ObjNum 7 snap 0 seq_num 7) 2023-06-27T12:38:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:73: read oid 27 snap 0 2023-06-27T12:38:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:73: expect (ObjNum 26 snap 0 seq_num 26) 2023-06-27T12:38:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:74: snap_create 2023-06-27T12:38:08.459 INFO:tasks.rados.rados.0.smithi138.stdout:68: expect (ObjNum 12 snap 0 seq_num 12) 2023-06-27T12:38:08.547 INFO:tasks.rados.rados.0.smithi138.stdout:66: expect (ObjNum 47 snap 0 seq_num 47) 2023-06-27T12:38:08.650 INFO:tasks.rados.rados.0.smithi138.stdout:71: expect (ObjNum 49 snap 0 seq_num 49) 2023-06-27T12:38:08.753 INFO:tasks.rados.rados.0.smithi138.stdout:73: expect (ObjNum 26 snap 0 seq_num 26) 2023-06-27T12:38:08.843 INFO:tasks.rados.rados.0.smithi138.stdout:65: finishing write tid 1 to smithi138138488-40 2023-06-27T12:38:08.843 INFO:tasks.rados.rados.0.smithi138.stdout:70: expect (ObjNum 35 snap 0 seq_num 35) 2023-06-27T12:38:08.939 INFO:tasks.rados.rados.0.smithi138.stdout:65: finishing write tid 2 to smithi138138488-40 2023-06-27T12:38:08.939 INFO:tasks.rados.rados.0.smithi138.stdout:65: finishing write tid 3 to smithi138138488-40 2023-06-27T12:38:08.939 INFO:tasks.rados.rados.0.smithi138.stdout:65: finishing write tid 4 to smithi138138488-40 2023-06-27T12:38:08.939 INFO:tasks.rados.rados.0.smithi138.stdout:65: finishing write tid 5 to smithi138138488-40 2023-06-27T12:38:08.940 INFO:tasks.rados.rados.0.smithi138.stdout:65: oid 40 updating version 0 to 14 2023-06-27T12:38:08.940 INFO:tasks.rados.rados.0.smithi138.stdout:65: oid 40 version 14 is already newer than 14 2023-06-27T12:38:08.940 INFO:tasks.rados.rados.0.smithi138.stdout:65: oid 40 version 14 is already newer than 14 2023-06-27T12:38:08.940 INFO:tasks.rados.rados.0.smithi138.stdout:65: oid 40 version 14 is already newer than 14 2023-06-27T12:38:08.940 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 14 (ObjNum 60 snap 1 seq_num 60) dirty exists 2023-06-27T12:38:08.940 INFO:tasks.rados.rados.0.smithi138.stdout:65: left oid 40 (ObjNum 60 snap 1 seq_num 60) 2023-06-27T12:38:08.941 INFO:tasks.rados.rados.0.smithi138.stdout:67: expect (ObjNum 27 snap 0 seq_num 27) 2023-06-27T12:38:09.029 INFO:tasks.rados.rados.0.smithi138.stdout:72: expect (ObjNum 7 snap 0 seq_num 7) 2023-06-27T12:38:09.108 INFO:tasks.rados.rados.0.smithi138.stdout:69: expect (ObjNum 42 snap 0 seq_num 42) 2023-06-27T12:38:09.405 INFO:tasks.rados.rados.0.smithi138.stdout:65: done (9 left) 2023-06-27T12:38:09.405 INFO:tasks.rados.rados.0.smithi138.stdout:66: done (8 left) 2023-06-27T12:38:09.406 INFO:tasks.rados.rados.0.smithi138.stdout:67: done (7 left) 2023-06-27T12:38:09.406 INFO:tasks.rados.rados.0.smithi138.stdout:68: done (6 left) 2023-06-27T12:38:09.406 INFO:tasks.rados.rados.0.smithi138.stdout:69: done (5 left) 2023-06-27T12:38:09.406 INFO:tasks.rados.rados.0.smithi138.stdout:70: done (4 left) 2023-06-27T12:38:09.406 INFO:tasks.rados.rados.0.smithi138.stdout:71: done (3 left) 2023-06-27T12:38:09.406 INFO:tasks.rados.rados.0.smithi138.stdout:72: done (2 left) 2023-06-27T12:38:09.406 INFO:tasks.rados.rados.0.smithi138.stdout:73: done (1 left) 2023-06-27T12:38:09.407 INFO:tasks.rados.rados.0.smithi138.stdout:74: done (0 left) 2023-06-27T12:38:09.407 INFO:tasks.rados.rados.0.smithi138.stdout:75: write oid 45 current snap is 2 2023-06-27T12:38:09.407 INFO:tasks.rados.rados.0.smithi138.stdout:75: seq_num 61 ranges {625227=721013,1952931=704669} 2023-06-27T12:38:09.412 INFO:tasks.rados.rados.0.smithi138.stdout:75: writing smithi138138488-45 from 625227 to 1346240 tid 1 2023-06-27T12:38:09.419 INFO:tasks.rados.rados.0.smithi138.stdout:75: writing smithi138138488-45 from 1952931 to 2657600 tid 2 2023-06-27T12:38:09.419 INFO:tasks.rados.rados.0.smithi138.stdout:76: read oid 16 snap -1 2023-06-27T12:38:09.420 INFO:tasks.rados.rados.0.smithi138.stdout:76: expect (ObjNum 15 snap 0 seq_num 15) 2023-06-27T12:38:09.420 INFO:tasks.rados.rados.0.smithi138.stdout:77: write oid 46 current snap is 2 2023-06-27T12:38:09.420 INFO:tasks.rados.rados.0.smithi138.stdout:77: seq_num 62 ranges {642034=796262,1895060=564680} 2023-06-27T12:38:09.429 INFO:tasks.rados.rados.0.smithi138.stdout:77: writing smithi138138488-46 from 642034 to 1438296 tid 1 2023-06-27T12:38:09.438 INFO:tasks.rados.rados.0.smithi138.stdout:77: writing smithi138138488-46 from 1895060 to 2459740 tid 2 2023-06-27T12:38:09.438 INFO:tasks.rados.rados.0.smithi138.stdout:78: read oid 26 snap 0 2023-06-27T12:38:09.439 INFO:tasks.rados.rados.0.smithi138.stdout:78: expect (ObjNum 25 snap 0 seq_num 25) 2023-06-27T12:38:09.439 INFO:tasks.rados.rados.0.smithi138.stdout:79: read oid 7 snap -1 2023-06-27T12:38:09.439 INFO:tasks.rados.rados.0.smithi138.stdout:79: expect (ObjNum 53 snap 0 seq_num 53) 2023-06-27T12:38:09.439 INFO:tasks.rados.rados.0.smithi138.stdout:80: write oid 44 current snap is 2 2023-06-27T12:38:09.439 INFO:tasks.rados.rados.0.smithi138.stdout:80: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2023-06-27T12:38:09.446 INFO:tasks.rados.rados.0.smithi138.stdout:80: writing smithi138138488-44 from 658841 to 1130352 tid 1 2023-06-27T12:38:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:80: writing smithi138138488-44 from 1553542 to 2261880 tid 2 2023-06-27T12:38:09.454 INFO:tasks.rados.rados.0.smithi138.stdout:80: writing smithi138138488-44 from 2964119 to 3058841 tid 3 2023-06-27T12:38:09.455 INFO:tasks.rados.rados.0.smithi138.stdout:81: read oid 30 snap -1 2023-06-27T12:38:09.455 INFO:tasks.rados.rados.0.smithi138.stdout:81: expect (ObjNum 29 snap 0 seq_num 29) 2023-06-27T12:38:09.455 INFO:tasks.rados.rados.0.smithi138.stdout:82: snap_remove snap 1 2023-06-27T12:38:09.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:09.748 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:09.743+0000 7fe741857700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:09.751 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:09.747+0000 7fe741857700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:10.416 INFO:tasks.rados.rados.0.smithi138.stdout:82: done (7 left) 2023-06-27T12:38:10.416 INFO:tasks.rados.rados.0.smithi138.stdout:83: delete oid 49 current snap is 2 2023-06-27T12:38:10.416 INFO:tasks.rados.rados.0.smithi138.stdout:76: expect (ObjNum 15 snap 0 seq_num 15) 2023-06-27T12:38:10.504 INFO:tasks.rados.rados.0.smithi138.stdout:75: finishing write tid 1 to smithi138138488-45 2023-06-27T12:38:10.504 INFO:tasks.rados.rados.0.smithi138.stdout:75: finishing write tid 2 to smithi138138488-45 2023-06-27T12:38:10.505 INFO:tasks.rados.rados.0.smithi138.stdout:79: expect (ObjNum 53 snap 0 seq_num 53) 2023-06-27T12:38:10.648 INFO:tasks.rados.rados.0.smithi138.stdout:81: expect (ObjNum 29 snap 0 seq_num 29) 2023-06-27T12:38:10.742 INFO:tasks.rados.rados.0.smithi138.stdout:75: finishing write tid 3 to smithi138138488-45 2023-06-27T12:38:10.742 INFO:tasks.rados.rados.0.smithi138.stdout:75: finishing write tid 4 to smithi138138488-45 2023-06-27T12:38:10.742 INFO:tasks.rados.rados.0.smithi138.stdout:75: oid 45 updating version 0 to 27 2023-06-27T12:38:10.742 INFO:tasks.rados.rados.0.smithi138.stdout:75: oid 45 version 27 is already newer than 27 2023-06-27T12:38:10.743 INFO:tasks.rados.rados.0.smithi138.stdout:75: oid 45 version 27 is already newer than 27 2023-06-27T12:38:10.743 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 27 (ObjNum 61 snap 2 seq_num 61) dirty exists 2023-06-27T12:38:10.743 INFO:tasks.rados.rados.0.smithi138.stdout:75: left oid 45 (ObjNum 61 snap 2 seq_num 61) 2023-06-27T12:38:10.743 INFO:tasks.rados.rados.0.smithi138.stdout:80: finishing write tid 1 to smithi138138488-44 2023-06-27T12:38:10.743 INFO:tasks.rados.rados.0.smithi138.stdout:75: done (7 left) 2023-06-27T12:38:10.743 INFO:tasks.rados.rados.0.smithi138.stdout:76: done (6 left) 2023-06-27T12:38:10.743 INFO:tasks.rados.rados.0.smithi138.stdout:79: done (5 left) 2023-06-27T12:38:10.743 INFO:tasks.rados.rados.0.smithi138.stdout:81: done (4 left) 2023-06-27T12:38:10.744 INFO:tasks.rados.rados.0.smithi138.stdout:83: done (3 left) 2023-06-27T12:38:10.744 INFO:tasks.rados.rados.0.smithi138.stdout:84: read oid 27 snap -1 2023-06-27T12:38:10.744 INFO:tasks.rados.rados.0.smithi138.stdout:84: expect (ObjNum 26 snap 0 seq_num 26) 2023-06-27T12:38:10.745 INFO:tasks.rados.rados.0.smithi138.stdout:85: read oid 29 snap 0 2023-06-27T12:38:10.745 INFO:tasks.rados.rados.0.smithi138.stdout:85: expect (ObjNum 28 snap 0 seq_num 28) 2023-06-27T12:38:10.745 INFO:tasks.rados.rados.0.smithi138.stdout:86: write oid 30 current snap is 2 2023-06-27T12:38:10.745 INFO:tasks.rados.rados.0.smithi138.stdout:78: expect (ObjNum 25 snap 0 seq_num 25) 2023-06-27T12:38:10.839 INFO:tasks.rados.rados.0.smithi138.stdout:77: finishing write tid 1 to smithi138138488-46 2023-06-27T12:38:10.839 INFO:tasks.rados.rados.0.smithi138.stdout:86: seq_num 65 ranges {692455=622009,2070506=712007} 2023-06-27T12:38:10.846 INFO:tasks.rados.rados.0.smithi138.stdout:86: writing smithi138138488-30 from 692455 to 1314464 tid 1 2023-06-27T12:38:10.855 INFO:tasks.rados.rados.0.smithi138.stdout:86: writing smithi138138488-30 from 2070506 to 2782513 tid 2 2023-06-27T12:38:10.855 INFO:tasks.rados.rados.0.smithi138.stdout:78: done (5 left) 2023-06-27T12:38:10.855 INFO:tasks.rados.rados.0.smithi138.stdout:87: snap_create 2023-06-27T12:38:10.855 INFO:tasks.rados.rados.0.smithi138.stdout:77: finishing write tid 2 to smithi138138488-46 2023-06-27T12:38:10.855 INFO:tasks.rados.rados.0.smithi138.stdout:77: finishing write tid 3 to smithi138138488-46 2023-06-27T12:38:10.855 INFO:tasks.rados.rados.0.smithi138.stdout:77: finishing write tid 4 to smithi138138488-46 2023-06-27T12:38:10.856 INFO:tasks.rados.rados.0.smithi138.stdout:77: oid 46 updating version 0 to 14 2023-06-27T12:38:10.856 INFO:tasks.rados.rados.0.smithi138.stdout:77: oid 46 version 14 is already newer than 14 2023-06-27T12:38:10.856 INFO:tasks.rados.rados.0.smithi138.stdout:77: oid 46 version 14 is already newer than 14 2023-06-27T12:38:10.856 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 14 (ObjNum 62 snap 2 seq_num 62) dirty exists 2023-06-27T12:38:10.856 INFO:tasks.rados.rados.0.smithi138.stdout:77: left oid 46 (ObjNum 62 snap 2 seq_num 62) 2023-06-27T12:38:10.856 INFO:tasks.rados.rados.0.smithi138.stdout:80: finishing write tid 2 to smithi138138488-44 2023-06-27T12:38:10.856 INFO:tasks.rados.rados.0.smithi138.stdout:80: finishing write tid 3 to smithi138138488-44 2023-06-27T12:38:10.857 INFO:tasks.rados.rados.0.smithi138.stdout:80: finishing write tid 4 to smithi138138488-44 2023-06-27T12:38:10.857 INFO:tasks.rados.rados.0.smithi138.stdout:80: finishing write tid 5 to smithi138138488-44 2023-06-27T12:38:10.857 INFO:tasks.rados.rados.0.smithi138.stdout:80: oid 44 updating version 0 to 20 2023-06-27T12:38:10.857 INFO:tasks.rados.rados.0.smithi138.stdout:80: oid 44 version 20 is already newer than 20 2023-06-27T12:38:10.857 INFO:tasks.rados.rados.0.smithi138.stdout:80: oid 44 version 20 is already newer than 20 2023-06-27T12:38:10.857 INFO:tasks.rados.rados.0.smithi138.stdout:80: oid 44 version 20 is already newer than 20 2023-06-27T12:38:10.857 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 20 (ObjNum 63 snap 2 seq_num 63) dirty exists 2023-06-27T12:38:10.857 INFO:tasks.rados.rados.0.smithi138.stdout:80: left oid 44 (ObjNum 63 snap 2 seq_num 63) 2023-06-27T12:38:10.858 INFO:tasks.rados.rados.0.smithi138.stdout:84: expect (ObjNum 26 snap 0 seq_num 26) 2023-06-27T12:38:10.948 INFO:tasks.rados.rados.0.smithi138.stdout:85: expect (ObjNum 28 snap 0 seq_num 28) 2023-06-27T12:38:11.040 INFO:tasks.rados.rados.0.smithi138.stdout:86: finishing write tid 1 to smithi138138488-30 2023-06-27T12:38:11.040 INFO:tasks.rados.rados.0.smithi138.stdout:86: finishing write tid 2 to smithi138138488-30 2023-06-27T12:38:11.041 INFO:tasks.rados.rados.0.smithi138.stdout:86: finishing write tid 3 to smithi138138488-30 2023-06-27T12:38:11.041 INFO:tasks.rados.rados.0.smithi138.stdout:86: finishing write tid 4 to smithi138138488-30 2023-06-27T12:38:11.041 INFO:tasks.rados.rados.0.smithi138.stdout:86: oid 30 updating version 0 to 16 2023-06-27T12:38:11.041 INFO:tasks.rados.rados.0.smithi138.stdout:86: oid 30 version 16 is already newer than 16 2023-06-27T12:38:11.041 INFO:tasks.rados.rados.0.smithi138.stdout:86: oid 30 version 16 is already newer than 16 2023-06-27T12:38:11.041 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 16 (ObjNum 65 snap 2 seq_num 65) dirty exists 2023-06-27T12:38:11.041 INFO:tasks.rados.rados.0.smithi138.stdout:86: left oid 30 (ObjNum 65 snap 2 seq_num 65) 2023-06-27T12:38:11.416 INFO:tasks.rados.rados.0.smithi138.stdout:77: done (5 left) 2023-06-27T12:38:11.417 INFO:tasks.rados.rados.0.smithi138.stdout:80: done (4 left) 2023-06-27T12:38:11.417 INFO:tasks.rados.rados.0.smithi138.stdout:84: done (3 left) 2023-06-27T12:38:11.417 INFO:tasks.rados.rados.0.smithi138.stdout:85: done (2 left) 2023-06-27T12:38:11.418 INFO:tasks.rados.rados.0.smithi138.stdout:86: done (1 left) 2023-06-27T12:38:11.418 INFO:tasks.rados.rados.0.smithi138.stdout:87: done (0 left) 2023-06-27T12:38:11.418 INFO:tasks.rados.rados.0.smithi138.stdout:88: write oid 6 current snap is 3 2023-06-27T12:38:11.419 INFO:tasks.rados.rados.0.smithi138.stdout:88: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2023-06-27T12:38:11.423 INFO:tasks.rados.rados.0.smithi138.stdout:88: writing smithi138138488-6 from 709262 to 1406520 tid 1 2023-06-27T12:38:11.430 INFO:tasks.rados.rados.0.smithi138.stdout:88: writing smithi138138488-6 from 2012635 to 2584653 tid 2 2023-06-27T12:38:11.433 INFO:tasks.rados.rados.0.smithi138.stdout:88: writing smithi138138488-6 from 3046388 to 3109262 tid 3 2023-06-27T12:38:11.433 INFO:tasks.rados.rados.0.smithi138.stdout:89: write oid 7 current snap is 3 2023-06-27T12:38:11.433 INFO:tasks.rados.rados.0.smithi138.stdout:89: seq_num 67 ranges {726069=772507,2071117=715676} 2023-06-27T12:38:11.441 INFO:tasks.rados.rados.0.smithi138.stdout:89: writing smithi138138488-7 from 726069 to 1498576 tid 1 2023-06-27T12:38:11.449 INFO:tasks.rados.rados.0.smithi138.stdout:89: writing smithi138138488-7 from 2071117 to 2786793 tid 2 2023-06-27T12:38:11.450 INFO:tasks.rados.rados.0.smithi138.stdout:90: write oid 22 current snap is 3 2023-06-27T12:38:11.450 INFO:tasks.rados.rados.0.smithi138.stdout:90: seq_num 68 ranges {742876=447756,1729599=575687,3101234=41642} 2023-06-27T12:38:11.457 INFO:tasks.rados.rados.0.smithi138.stdout:90: writing smithi138138488-22 from 742876 to 1190632 tid 1 2023-06-27T12:38:11.465 INFO:tasks.rados.rados.0.smithi138.stdout:90: writing smithi138138488-22 from 1729599 to 2305286 tid 2 2023-06-27T12:38:11.470 INFO:tasks.rados.rados.0.smithi138.stdout:90: writing smithi138138488-22 from 3101234 to 3142876 tid 3 2023-06-27T12:38:11.470 INFO:tasks.rados.rados.0.smithi138.stdout:91: delete oid 14 current snap is 3 2023-06-27T12:38:11.546 INFO:tasks.rados.rados.0.smithi138.stdout:91: done (3 left) 2023-06-27T12:38:11.546 INFO:tasks.rados.rados.0.smithi138.stdout:92: write (excl) oid 45 current snap is 3 2023-06-27T12:38:11.546 INFO:tasks.rados.rados.0.smithi138.stdout:92: seq_num 70 ranges {776490=714607,2246563=579356} 2023-06-27T12:38:11.554 INFO:tasks.rados.rados.0.smithi138.stdout:92: writing smithi138138488-45 from 776490 to 1491097 tid 1 2023-06-27T12:38:11.562 INFO:tasks.rados.rados.0.smithi138.stdout:92: writing smithi138138488-45 from 2246563 to 2825919 tid 2 2023-06-27T12:38:11.562 INFO:tasks.rados.rados.0.smithi138.stdout:93: write oid 27 current snap is 3 2023-06-27T12:38:11.563 INFO:tasks.rados.rados.0.smithi138.stdout:93: seq_num 71 ranges {793297=789856,2305045=723014} 2023-06-27T12:38:11.571 INFO:tasks.rados.rados.0.smithi138.stdout:93: writing smithi138138488-27 from 793297 to 1583153 tid 1 2023-06-27T12:38:11.580 INFO:tasks.rados.rados.0.smithi138.stdout:93: writing smithi138138488-27 from 2305045 to 3028059 tid 2 2023-06-27T12:38:11.580 INFO:tasks.rados.rados.0.smithi138.stdout:94: read oid 14 snap -1 2023-06-27T12:38:11.580 INFO:tasks.rados.rados.0.smithi138.stdout:94: expect deleted 2023-06-27T12:38:11.580 INFO:tasks.rados.rados.0.smithi138.stdout:95: read oid 4 snap -1 2023-06-27T12:38:11.581 INFO:tasks.rados.rados.0.smithi138.stdout:95: expect (ObjNum 3 snap 0 seq_num 3) 2023-06-27T12:38:11.581 INFO:tasks.rados.rados.0.smithi138.stdout:96: delete oid 49 current snap is 3 2023-06-27T12:38:11.581 INFO:tasks.rados.rados.0.smithi138.stdout:89: finishing write tid 1 to smithi138138488-7 2023-06-27T12:38:11.581 INFO:tasks.rados.rados.0.smithi138.stdout:89: finishing write tid 2 to smithi138138488-7 2023-06-27T12:38:11.581 INFO:tasks.rados.rados.0.smithi138.stdout:89: finishing write tid 3 to smithi138138488-7 2023-06-27T12:38:11.582 INFO:tasks.rados.rados.0.smithi138.stdout:89: finishing write tid 4 to smithi138138488-7 2023-06-27T12:38:11.582 INFO:tasks.rados.rados.0.smithi138.stdout:89: oid 7 updating version 0 to 16 2023-06-27T12:38:11.582 INFO:tasks.rados.rados.0.smithi138.stdout:89: oid 7 version 16 is already newer than 16 2023-06-27T12:38:11.582 INFO:tasks.rados.rados.0.smithi138.stdout:89: oid 7 version 16 is already newer than 16 2023-06-27T12:38:11.582 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 16 (ObjNum 67 snap 3 seq_num 67) dirty exists 2023-06-27T12:38:11.582 INFO:tasks.rados.rados.0.smithi138.stdout:89: left oid 7 (ObjNum 67 snap 3 seq_num 67) 2023-06-27T12:38:11.582 INFO:tasks.rados.rados.0.smithi138.stdout:88: finishing write tid 1 to smithi138138488-6 2023-06-27T12:38:11.583 INFO:tasks.rados.rados.0.smithi138.stdout:90: finishing write tid 1 to smithi138138488-22 2023-06-27T12:38:11.583 INFO:tasks.rados.rados.0.smithi138.stdout:88: finishing write tid 2 to smithi138138488-6 2023-06-27T12:38:11.598 INFO:tasks.rados.rados.0.smithi138.stdout:95: expect (ObjNum 3 snap 0 seq_num 3) 2023-06-27T12:38:11.682 INFO:tasks.rados.rados.0.smithi138.stdout:88: finishing write tid 3 to smithi138138488-6 2023-06-27T12:38:11.682 INFO:tasks.rados.rados.0.smithi138.stdout:89: done (7 left) 2023-06-27T12:38:11.682 INFO:tasks.rados.rados.0.smithi138.stdout:94: done (6 left) 2023-06-27T12:38:11.683 INFO:tasks.rados.rados.0.smithi138.stdout:95: done (5 left) 2023-06-27T12:38:11.683 INFO:tasks.rados.rados.0.smithi138.stdout:96: done (4 left) 2023-06-27T12:38:11.683 INFO:tasks.rados.rados.0.smithi138.stdout:97: snap_remove snap 0 2023-06-27T12:38:11.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:11.946 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:11.941+0000 7fa027eef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:11.949 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:11.945+0000 7fa027eef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:12.420 INFO:tasks.rados.rados.0.smithi138.stdout:97: done (4 left) 2023-06-27T12:38:12.421 INFO:tasks.rados.rados.0.smithi138.stdout:98: write (excl) oid 24 current snap is 3 2023-06-27T12:38:12.421 INFO:tasks.rados.rados.0.smithi138.stdout:90: finishing write tid 2 to smithi138138488-22 2023-06-27T12:38:12.421 INFO:tasks.rados.rados.0.smithi138.stdout:98: seq_num 73 ranges {426911=540354,1622009=726683,3121996=104915} 2023-06-27T12:38:12.425 INFO:tasks.rados.rados.0.smithi138.stdout:98: writing smithi138138488-24 from 426911 to 967265 tid 1 2023-06-27T12:38:12.434 INFO:tasks.rados.rados.0.smithi138.stdout:98: writing smithi138138488-24 from 1622009 to 2348692 tid 2 2023-06-27T12:38:12.440 INFO:tasks.rados.rados.0.smithi138.stdout:98: writing smithi138138488-24 from 3121996 to 3226911 tid 3 2023-06-27T12:38:12.440 INFO:tasks.rados.rados.0.smithi138.stdout:99: snap_remove snap 2 2023-06-27T12:38:13.403 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:13.404 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:13.404 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:13.428 INFO:tasks.rados.rados.0.smithi138.stdout:99: done (5 left) 2023-06-27T12:38:13.428 INFO:tasks.rados.rados.0.smithi138.stdout:100: write oid 49 current snap is 3 2023-06-27T12:38:13.428 INFO:tasks.rados.rados.0.smithi138.stdout:88: finishing write tid 4 to smithi138138488-6 2023-06-27T12:38:13.429 INFO:tasks.rados.rados.0.smithi138.stdout:100: seq_num 74 ranges {443718=615603,1564138=470341,2633066=610652} 2023-06-27T12:38:13.434 INFO:tasks.rados.rados.0.smithi138.stdout:100: writing smithi138138488-49 from 443718 to 1059321 tid 1 2023-06-27T12:38:13.440 INFO:tasks.rados.rados.0.smithi138.stdout:100: writing smithi138138488-49 from 1564138 to 2034479 tid 2 2023-06-27T12:38:13.447 INFO:tasks.rados.rados.0.smithi138.stdout:100: writing smithi138138488-49 from 2633066 to 3243718 tid 3 2023-06-27T12:38:13.447 INFO:tasks.rados.rados.0.smithi138.stdout:101: read oid 8 snap -1 2023-06-27T12:38:13.447 INFO:tasks.rados.rados.0.smithi138.stdout:101: expect (ObjNum 7 snap 0 seq_num 7) 2023-06-27T12:38:13.447 INFO:tasks.rados.rados.0.smithi138.stdout:102: read oid 32 snap -1 2023-06-27T12:38:13.448 INFO:tasks.rados.rados.0.smithi138.stdout:102: expect (ObjNum 31 snap 0 seq_num 31) 2023-06-27T12:38:13.448 INFO:tasks.rados.rados.0.smithi138.stdout:103: read oid 30 snap -1 2023-06-27T12:38:13.448 INFO:tasks.rados.rados.0.smithi138.stdout:103: expect (ObjNum 65 snap 2 seq_num 65) 2023-06-27T12:38:13.449 INFO:tasks.rados.rados.0.smithi138.stdout:104: read oid 2 snap -1 2023-06-27T12:38:13.449 INFO:tasks.rados.rados.0.smithi138.stdout:104: expect (ObjNum 50 snap 0 seq_num 50) 2023-06-27T12:38:13.449 INFO:tasks.rados.rados.0.smithi138.stdout:105: read oid 50 snap -1 2023-06-27T12:38:13.449 INFO:tasks.rados.rados.0.smithi138.stdout:105: expect (ObjNum 49 snap 0 seq_num 49) 2023-06-27T12:38:13.449 INFO:tasks.rados.rados.0.smithi138.stdout:106: read oid 41 snap -1 2023-06-27T12:38:13.449 INFO:tasks.rados.rados.0.smithi138.stdout:106: expect (ObjNum 40 snap 0 seq_num 40) 2023-06-27T12:38:13.449 INFO:tasks.rados.rados.0.smithi138.stdout:107: delete oid 17 current snap is 3 2023-06-27T12:38:13.449 INFO:tasks.rados.rados.0.smithi138.stdout:88: finishing write tid 5 to smithi138138488-6 2023-06-27T12:38:13.450 INFO:tasks.rados.rados.0.smithi138.stdout:88: oid 6 updating version 0 to 19 2023-06-27T12:38:13.450 INFO:tasks.rados.rados.0.smithi138.stdout:88: oid 6 version 19 is already newer than 19 2023-06-27T12:38:13.450 INFO:tasks.rados.rados.0.smithi138.stdout:88: oid 6 version 19 is already newer than 19 2023-06-27T12:38:13.450 INFO:tasks.rados.rados.0.smithi138.stdout:88: oid 6 version 19 is already newer than 19 2023-06-27T12:38:13.450 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 19 (ObjNum 66 snap 3 seq_num 66) dirty exists 2023-06-27T12:38:13.450 INFO:tasks.rados.rados.0.smithi138.stdout:88: left oid 6 (ObjNum 66 snap 3 seq_num 66) 2023-06-27T12:38:13.450 INFO:tasks.rados.rados.0.smithi138.stdout:90: finishing write tid 3 to smithi138138488-22 2023-06-27T12:38:13.451 INFO:tasks.rados.rados.0.smithi138.stdout:92: finishing write tid 1 to smithi138138488-45 2023-06-27T12:38:13.451 INFO:tasks.rados.rados.0.smithi138.stdout:90: finishing write tid 4 to smithi138138488-22 2023-06-27T12:38:13.451 INFO:tasks.rados.rados.0.smithi138.stdout:90: finishing write tid 5 to smithi138138488-22 2023-06-27T12:38:13.451 INFO:tasks.rados.rados.0.smithi138.stdout:90: oid 22 updating version 0 to 16 2023-06-27T12:38:13.451 INFO:tasks.rados.rados.0.smithi138.stdout:90: oid 22 version 16 is already newer than 16 2023-06-27T12:38:13.451 INFO:tasks.rados.rados.0.smithi138.stdout:90: oid 22 version 16 is already newer than 16 2023-06-27T12:38:13.451 INFO:tasks.rados.rados.0.smithi138.stdout:90: oid 22 version 16 is already newer than 16 2023-06-27T12:38:13.451 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 16 (ObjNum 68 snap 3 seq_num 68) dirty exists 2023-06-27T12:38:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:90: left oid 22 (ObjNum 68 snap 3 seq_num 68) 2023-06-27T12:38:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:93: finishing write tid 1 to smithi138138488-27 2023-06-27T12:38:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:93: finishing write tid 2 to smithi138138488-27 2023-06-27T12:38:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:93: finishing write tid 3 to smithi138138488-27 2023-06-27T12:38:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:93: finishing write tid 4 to smithi138138488-27 2023-06-27T12:38:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:93: oid 27 updating version 0 to 16 2023-06-27T12:38:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:93: oid 27 version 16 is already newer than 16 2023-06-27T12:38:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:93: oid 27 version 16 is already newer than 16 2023-06-27T12:38:13.453 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 16 (ObjNum 71 snap 3 seq_num 71) dirty exists 2023-06-27T12:38:13.453 INFO:tasks.rados.rados.0.smithi138.stdout:93: left oid 27 (ObjNum 71 snap 3 seq_num 71) 2023-06-27T12:38:13.453 INFO:tasks.rados.rados.0.smithi138.stdout:92: finishing write tid 2 to smithi138138488-45 2023-06-27T12:38:13.453 INFO:tasks.rados.rados.0.smithi138.stdout:92: finishing write tid 3 to smithi138138488-45 2023-06-27T12:38:13.453 INFO:tasks.rados.rados.0.smithi138.stdout:92: finishing write tid 4 to smithi138138488-45 2023-06-27T12:38:13.453 INFO:tasks.rados.rados.0.smithi138.stdout:92: oid 45 updating version 0 to 30 2023-06-27T12:38:13.453 INFO:tasks.rados.rados.0.smithi138.stdout:92: oid 45 updating version 30 to 33 2023-06-27T12:38:13.453 INFO:tasks.rados.rados.0.smithi138.stdout:92: oid 45 version 33 is already newer than 33 2023-06-27T12:38:13.454 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 33 (ObjNum 70 snap 3 seq_num 70) dirty exists 2023-06-27T12:38:13.454 INFO:tasks.rados.rados.0.smithi138.stdout:92: left oid 45 (ObjNum 70 snap 3 seq_num 70) 2023-06-27T12:38:13.454 INFO:tasks.rados.rados.0.smithi138.stdout:98: finishing write tid 1 to smithi138138488-24 2023-06-27T12:38:13.454 INFO:tasks.rados.rados.0.smithi138.stdout:98: finishing write tid 2 to smithi138138488-24 2023-06-27T12:38:13.454 INFO:tasks.rados.rados.0.smithi138.stdout:98: finishing write tid 3 to smithi138138488-24 2023-06-27T12:38:13.454 INFO:tasks.rados.rados.0.smithi138.stdout:98: finishing write tid 4 to smithi138138488-24 2023-06-27T12:38:13.454 INFO:tasks.rados.rados.0.smithi138.stdout:98: finishing write tid 5 to smithi138138488-24 2023-06-27T12:38:13.454 INFO:tasks.rados.rados.0.smithi138.stdout:98: oid 24 updating version 0 to 27 2023-06-27T12:38:13.455 INFO:tasks.rados.rados.0.smithi138.stdout:98: oid 24 version 27 is already newer than 23 2023-06-27T12:38:13.455 INFO:tasks.rados.rados.0.smithi138.stdout:98: oid 24 version 27 is already newer than 27 2023-06-27T12:38:13.455 INFO:tasks.rados.rados.0.smithi138.stdout:98: oid 24 version 27 is already newer than 27 2023-06-27T12:38:13.455 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 27 (ObjNum 73 snap 3 seq_num 73) dirty exists 2023-06-27T12:38:13.455 INFO:tasks.rados.rados.0.smithi138.stdout:98: left oid 24 (ObjNum 73 snap 3 seq_num 73) 2023-06-27T12:38:13.526 INFO:tasks.rados.rados.0.smithi138.stdout:102: expect (ObjNum 31 snap 0 seq_num 31) 2023-06-27T12:38:13.608 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:13.599+0000 7f471cf85700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:13.608+0000 7f471cf85700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:13.638 INFO:tasks.rados.rados.0.smithi138.stdout:104: expect (ObjNum 50 snap 0 seq_num 50) 2023-06-27T12:38:13.779 INFO:tasks.rados.rados.0.smithi138.stdout:106: expect (ObjNum 40 snap 0 seq_num 40) 2023-06-27T12:38:13.821 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:13.822 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:13.833 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-06-27T12:38:13.423899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117363853,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":253,"num_read_kb":43720,"num_write":370,"num_write_kb":65071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":276908,"kb_used_data":164256,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377210452,"statfs":{"total":386547056640,"available":386263502848,"internally_reserved":0,"allocated":168198144,"data_stored":168361034,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022680"},"pg_stats":[{"pgid":"3.f","version":"18'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.440542+0000","last_change":"2023-06-27T12:38:04.542053+0000","last_active":"2023-06-27T12:38:08.440542+0000","last_peered":"2023-06-27T12:38:08.440542+0000","last_clean":"2023-06-27T12:38:08.440542+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:08.440542+0000","last_undegraded":"2023-06-27T12:38:08.440542+0000","last_fullsized":"2023-06-27T12:38:08.440542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"19'12","reported_seq":35,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.462725+0000","last_change":"2023-06-27T12:38:04.583831+0000","last_active":"2023-06-27T12:38:08.462725+0000","last_peered":"2023-06-27T12:38:08.462725+0000","last_clean":"2023-06-27T12:38:08.462725+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:08.462725+0000","last_undegraded":"2023-06-27T12:38:08.462725+0000","last_fullsized":"2023-06-27T12:38:08.462725+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9798349,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":7050,"num_write":16,"num_write_kb":4223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"18'23","reported_seq":46,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.488664+0000","last_change":"2023-06-27T12:38:04.533151+0000","last_active":"2023-06-27T12:38:08.488664+0000","last_peered":"2023-06-27T12:38:08.488664+0000","last_clean":"2023-06-27T12:38:08.488664+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:08.488664+0000","last_undegraded":"2023-06-27T12:38:08.488664+0000","last_fullsized":"2023-06-27T12:38:08.488664+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12773155,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":4857,"num_write":30,"num_write_kb":8165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"20'14","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.484278+0000","last_change":"2023-06-27T12:38:04.527650+0000","last_active":"2023-06-27T12:38:08.484278+0000","last_peered":"2023-06-27T12:38:08.484278+0000","last_clean":"2023-06-27T12:38:08.484278+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:08.484278+0000","last_undegraded":"2023-06-27T12:38:08.484278+0000","last_fullsized":"2023-06-27T12:38:08.484278+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8235331,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":17,"num_write_kb":4732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"18'6","reported_seq":25,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.437625+0000","last_change":"2023-06-27T12:38:04.575319+0000","last_active":"2023-06-27T12:38:08.437625+0000","last_peered":"2023-06-27T12:38:08.437625+0000","last_clean":"2023-06-27T12:38:08.437625+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:08.437625+0000","last_undegraded":"2023-06-27T12:38:08.437625+0000","last_fullsized":"2023-06-27T12:38:08.437625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4352947,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":7285,"num_write":8,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"18'15","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.415026+0000","last_change":"2023-06-27T12:38:04.536047+0000","last_active":"2023-06-27T12:38:08.415026+0000","last_peered":"2023-06-27T12:38:08.415026+0000","last_clean":"2023-06-27T12:38:08.415026+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:08.415026+0000","last_undegraded":"2023-06-27T12:38:08.415026+0000","last_fullsized":"2023-06-27T12:38:08.415026+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12420208,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":5,"num_write":20,"num_write_kb":5389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"18'12","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.505725+0000","last_change":"2023-06-27T12:38:04.552558+0000","last_active":"2023-06-27T12:38:08.505725+0000","last_peered":"2023-06-27T12:38:08.505725+0000","last_clean":"2023-06-27T12:38:08.505725+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:08.505725+0000","last_undegraded":"2023-06-27T12:38:08.505725+0000","last_fullsized":"2023-06-27T12:38:08.505725+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7495823,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3685,"num_write":16,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"18'3","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.461143+0000","last_change":"2023-06-27T12:38:04.583941+0000","last_active":"2023-06-27T12:38:08.461143+0000","last_peered":"2023-06-27T12:38:08.461143+0000","last_clean":"2023-06-27T12:38:08.461143+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:08.461143+0000","last_undegraded":"2023-06-27T12:38:08.461143+0000","last_fullsized":"2023-06-27T12:38:08.461143+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2823543,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":5855,"num_write":4,"num_write_kb":1548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.436212+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:08.436212+0000","last_peered":"2023-06-27T12:38:08.436212+0000","last_clean":"2023-06-27T12:38:08.436212+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:08.436212+0000","last_undegraded":"2023-06-27T12:38:08.436212+0000","last_fullsized":"2023-06-27T12:38:08.436212+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.435406+0000","last_change":"2023-06-27T12:38:04.518757+0000","last_active":"2023-06-27T12:38:08.435406+0000","last_peered":"2023-06-27T12:38:08.435406+0000","last_clean":"2023-06-27T12:38:08.435406+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:08.435406+0000","last_undegraded":"2023-06-27T12:38:08.435406+0000","last_fullsized":"2023-06-27T12:38:08.435406+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.439192+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:08.439192+0000","last_peered":"2023-06-27T12:38:08.439192+0000","last_clean":"2023-06-27T12:38:08.439192+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:08.439192+0000","last_undegraded":"2023-06-27T12:38:08.439192+0000","last_fullsized":"2023-06-27T12:38:08.439192+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"19'11","reported_seq":29,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.436202+0000","last_change":"2023-06-27T12:38:04.559754+0000","last_active":"2023-06-27T12:38:08.436202+0000","last_peered":"2023-06-27T12:38:08.436202+0000","last_clean":"2023-06-27T12:38:08.436202+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:08.436202+0000","last_undegraded":"2023-06-27T12:38:08.436202+0000","last_fullsized":"2023-06-27T12:38:08.436202+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2084035,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":12,"num_write_kb":3321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.438260+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:08.438260+0000","last_peered":"2023-06-27T12:38:08.438260+0000","last_clean":"2023-06-27T12:38:08.438260+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:08.438260+0000","last_undegraded":"2023-06-27T12:38:08.438260+0000","last_fullsized":"2023-06-27T12:38:08.438260+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"18'23","reported_seq":45,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.431323+0000","last_change":"2023-06-27T12:38:04.594799+0000","last_active":"2023-06-27T12:38:08.431323+0000","last_peered":"2023-06-27T12:38:08.431323+0000","last_clean":"2023-06-27T12:38:08.431323+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:08.431323+0000","last_undegraded":"2023-06-27T12:38:08.431323+0000","last_fullsized":"2023-06-27T12:38:08.431323+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13243751,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":3283,"num_write":30,"num_write_kb":8534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.437593+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:08.437593+0000","last_peered":"2023-06-27T12:38:08.437593+0000","last_clean":"2023-06-27T12:38:08.437593+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:08.437593+0000","last_undegraded":"2023-06-27T12:38:08.437593+0000","last_fullsized":"2023-06-27T12:38:08.437593+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":148,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.422652+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:08.422652+0000","last_peered":"2023-06-27T12:38:08.422652+0000","last_clean":"2023-06-27T12:38:08.422652+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:08.422652+0000","last_undegraded":"2023-06-27T12:38:08.422652+0000","last_fullsized":"2023-06-27T12:38:08.422652+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"18'18","reported_seq":37,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.433878+0000","last_change":"2023-06-27T12:38:04.534416+0000","last_active":"2023-06-27T12:38:08.433878+0000","last_peered":"2023-06-27T12:38:08.433878+0000","last_clean":"2023-06-27T12:38:08.433878+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:08.433878+0000","last_undegraded":"2023-06-27T12:38:08.433878+0000","last_fullsized":"2023-06-27T12:38:08.433878+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11647086,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":6,"num_write":24,"num_write_kb":5604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.436530+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:08.436530+0000","last_peered":"2023-06-27T12:38:08.436530+0000","last_clean":"2023-06-27T12:38:08.436530+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:08.436530+0000","last_undegraded":"2023-06-27T12:38:08.436530+0000","last_fullsized":"2023-06-27T12:38:08.436530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"18'12","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.410217+0000","last_change":"2023-06-27T12:38:04.576963+0000","last_active":"2023-06-27T12:38:08.410217+0000","last_peered":"2023-06-27T12:38:08.410217+0000","last_clean":"2023-06-27T12:38:08.410217+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:08.410217+0000","last_undegraded":"2023-06-27T12:38:08.410217+0000","last_fullsized":"2023-06-27T12:38:08.410217+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9243718,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":3716,"num_write":16,"num_write_kb":4145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.406712+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:08.406712+0000","last_peered":"2023-06-27T12:38:08.406712+0000","last_clean":"2023-06-27T12:38:08.406712+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:08.406712+0000","last_undegraded":"2023-06-27T12:38:08.406712+0000","last_fullsized":"2023-06-27T12:38:08.406712+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"19'12","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.449146+0000","last_change":"2023-06-27T12:38:04.589224+0000","last_active":"2023-06-27T12:38:08.449146+0000","last_peered":"2023-06-27T12:38:08.449146+0000","last_clean":"2023-06-27T12:38:08.449146+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:08.449146+0000","last_undegraded":"2023-06-27T12:38:08.449146+0000","last_fullsized":"2023-06-27T12:38:08.449146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9831963,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":3806,"num_write":16,"num_write_kb":4475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"18'6","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.405931+0000","last_change":"2023-06-27T12:38:04.560017+0000","last_active":"2023-06-27T12:38:08.405931+0000","last_peered":"2023-06-27T12:38:08.405931+0000","last_clean":"2023-06-27T12:38:08.405931+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:08.405931+0000","last_undegraded":"2023-06-27T12:38:08.405931+0000","last_fullsized":"2023-06-27T12:38:08.405931+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4873964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":8,"num_write_kb":2305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.436457+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:08.436457+0000","last_peered":"2023-06-27T12:38:08.436457+0000","last_clean":"2023-06-27T12:38:08.436457+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:08.436457+0000","last_undegraded":"2023-06-27T12:38:08.436457+0000","last_fullsized":"2023-06-27T12:38:08.436457+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.441402+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:08.441402+0000","last_peered":"2023-06-27T12:38:08.441402+0000","last_clean":"2023-06-27T12:38:08.441402+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:08.441402+0000","last_undegraded":"2023-06-27T12:38:08.441402+0000","last_fullsized":"2023-06-27T12:38:08.441402+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"19'9","reported_seq":29,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-27T12:38:08.449847+0000","last_change":"2023-06-27T12:38:04.535828+0000","last_active":"2023-06-27T12:38:08.449847+0000","last_peered":"2023-06-27T12:38:08.449847+0000","last_clean":"2023-06-27T12:38:08.449847+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:08.449847+0000","last_undegraded":"2023-06-27T12:38:08.449847+0000","last_fullsized":"2023-06-27T12:38:08.449847+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7949612,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":4071,"num_write":12,"num_write_kb":3658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":116773485,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":43629,"num_write":233,"num_write_kb":63177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":180,"ondisk_log_size":180,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72612,"kb_used_data":45788,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94299228,"statfs":{"total":96636764160,"available":96562409472,"internally_reserved":0,"allocated":46886912,"data_stored":47026457,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":62816,"kb_used_data":35992,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94309024,"statfs":{"total":96636764160,"available":96572440576,"internally_reserved":0,"allocated":36855808,"data_stored":37070960,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705680,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":67032,"kb_used_data":38392,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94304808,"statfs":{"total":96636764160,"available":96568123392,"internally_reserved":0,"allocated":39313408,"data_stored":39496072,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74448,"kb_used_data":44084,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94297392,"statfs":{"total":96636764160,"available":96560529408,"internally_reserved":0,"allocated":45142016,"data_stored":44767545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:13.882 INFO:tasks.rados.rados.0.smithi138.stdout:101: expect (ObjNum 7 snap 0 seq_num 7) 2023-06-27T12:38:13.967 INFO:tasks.rados.rados.0.smithi138.stdout:105: expect (ObjNum 49 snap 0 seq_num 49) 2023-06-27T12:38:14.017 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:14.075 INFO:tasks.rados.rados.0.smithi138.stdout:103: expect (ObjNum 65 snap 2 seq_num 65) 2023-06-27T12:38:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:14.140+0000 7f2edeed5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:14.145+0000 7f2edeed5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:14.245 INFO:tasks.rados.rados.0.smithi138.stdout:100: finishing write tid 1 to smithi138138488-49 2023-06-27T12:38:14.245 INFO:tasks.rados.rados.0.smithi138.stdout:100: finishing write tid 2 to smithi138138488-49 2023-06-27T12:38:14.245 INFO:tasks.rados.rados.0.smithi138.stdout:100: finishing write tid 3 to smithi138138488-49 2023-06-27T12:38:14.246 INFO:tasks.rados.rados.0.smithi138.stdout:100: finishing write tid 4 to smithi138138488-49 2023-06-27T12:38:14.246 INFO:tasks.rados.rados.0.smithi138.stdout:100: finishing write tid 5 to smithi138138488-49 2023-06-27T12:38:14.246 INFO:tasks.rados.rados.0.smithi138.stdout:100: oid 49 updating version 0 to 25 2023-06-27T12:38:14.246 INFO:tasks.rados.rados.0.smithi138.stdout:100: oid 49 version 25 is already newer than 25 2023-06-27T12:38:14.246 INFO:tasks.rados.rados.0.smithi138.stdout:100: oid 49 version 25 is already newer than 25 2023-06-27T12:38:14.246 INFO:tasks.rados.rados.0.smithi138.stdout:100: oid 49 version 25 is already newer than 25 2023-06-27T12:38:14.246 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 49 v 25 (ObjNum 74 snap 3 seq_num 74) dirty exists 2023-06-27T12:38:14.246 INFO:tasks.rados.rados.0.smithi138.stdout:100: left oid 49 (ObjNum 74 snap 3 seq_num 74) 2023-06-27T12:38:14.247 INFO:tasks.rados.rados.0.smithi138.stdout:88: done (12 left) 2023-06-27T12:38:14.247 INFO:tasks.rados.rados.0.smithi138.stdout:90: done (11 left) 2023-06-27T12:38:14.247 INFO:tasks.rados.rados.0.smithi138.stdout:92: done (10 left) 2023-06-27T12:38:14.247 INFO:tasks.rados.rados.0.smithi138.stdout:93: done (9 left) 2023-06-27T12:38:14.247 INFO:tasks.rados.rados.0.smithi138.stdout:98: done (8 left) 2023-06-27T12:38:14.247 INFO:tasks.rados.rados.0.smithi138.stdout:100: done (7 left) 2023-06-27T12:38:14.247 INFO:tasks.rados.rados.0.smithi138.stdout:101: done (6 left) 2023-06-27T12:38:14.247 INFO:tasks.rados.rados.0.smithi138.stdout:102: done (5 left) 2023-06-27T12:38:14.248 INFO:tasks.rados.rados.0.smithi138.stdout:103: done (4 left) 2023-06-27T12:38:14.248 INFO:tasks.rados.rados.0.smithi138.stdout:104: done (3 left) 2023-06-27T12:38:14.248 INFO:tasks.rados.rados.0.smithi138.stdout:105: done (2 left) 2023-06-27T12:38:14.248 INFO:tasks.rados.rados.0.smithi138.stdout:106: done (1 left) 2023-06-27T12:38:14.248 INFO:tasks.rados.rados.0.smithi138.stdout:107: done (0 left) 2023-06-27T12:38:14.248 INFO:tasks.rados.rados.0.smithi138.stdout:108: snap_create 2023-06-27T12:38:14.431 INFO:tasks.rados.rados.0.smithi138.stdout:108: done (0 left) 2023-06-27T12:38:14.431 INFO:tasks.rados.rados.0.smithi138.stdout:109: delete oid 42 current snap is 4 2023-06-27T12:38:14.500 INFO:tasks.rados.rados.0.smithi138.stdout:109: done (0 left) 2023-06-27T12:38:14.500 INFO:tasks.rados.rados.0.smithi138.stdout:110: read oid 48 snap 3 2023-06-27T12:38:14.501 INFO:tasks.rados.rados.0.smithi138.stdout:110: expect (ObjNum 47 snap 0 seq_num 47) 2023-06-27T12:38:14.501 INFO:tasks.rados.rados.0.smithi138.stdout:111: snap_create 2023-06-27T12:38:14.520 INFO:tasks.rados.rados.0.smithi138.stdout:110: expect (ObjNum 47 snap 0 seq_num 47) 2023-06-27T12:38:15.432 INFO:tasks.rados.rados.0.smithi138.stdout:110: done (1 left) 2023-06-27T12:38:15.433 INFO:tasks.rados.rados.0.smithi138.stdout:111: done (0 left) 2023-06-27T12:38:15.433 INFO:tasks.rados.rados.0.smithi138.stdout:112: snap_create 2023-06-27T12:38:16.227 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:16.365 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:16.360+0000 7f8ed8ccc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:16.368 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:16.364+0000 7f8ed8ccc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:16.438 INFO:tasks.rados.rados.0.smithi138.stdout:112: done (0 left) 2023-06-27T12:38:16.439 INFO:tasks.rados.rados.0.smithi138.stdout:113: read oid 35 snap -1 2023-06-27T12:38:16.439 INFO:tasks.rados.rados.0.smithi138.stdout:113: expect (ObjNum 56 snap 0 seq_num 56) 2023-06-27T12:38:16.439 INFO:tasks.rados.rados.0.smithi138.stdout:114: read oid 9 snap -1 2023-06-27T12:38:16.439 INFO:tasks.rados.rados.0.smithi138.stdout:114: expect (ObjNum 8 snap 0 seq_num 8) 2023-06-27T12:38:16.439 INFO:tasks.rados.rados.0.smithi138.stdout:115: delete oid 26 current snap is 6 2023-06-27T12:38:16.478 INFO:tasks.rados.rados.0.smithi138.stdout:114: expect (ObjNum 8 snap 0 seq_num 8) 2023-06-27T12:38:16.561 INFO:tasks.rados.rados.0.smithi138.stdout:113: expect (ObjNum 56 snap 0 seq_num 56) 2023-06-27T12:38:16.710 INFO:tasks.rados.rados.0.smithi138.stdout:113: done (2 left) 2023-06-27T12:38:16.710 INFO:tasks.rados.rados.0.smithi138.stdout:114: done (1 left) 2023-06-27T12:38:16.710 INFO:tasks.rados.rados.0.smithi138.stdout:115: done (0 left) 2023-06-27T12:38:16.710 INFO:tasks.rados.rados.0.smithi138.stdout:116: write oid 28 current snap is 6 2023-06-27T12:38:16.710 INFO:tasks.rados.rados.0.smithi138.stdout:116: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2023-06-27T12:38:16.716 INFO:tasks.rados.rados.0.smithi138.stdout:116: writing smithi138138488-28 from 510946 to 1143898 tid 1 2023-06-27T12:38:16.722 INFO:tasks.rados.rados.0.smithi138.stdout:116: writing smithi138138488-28 from 1798066 to 2275745 tid 2 2023-06-27T12:38:16.728 INFO:tasks.rados.rados.0.smithi138.stdout:116: writing smithi138138488-28 from 2742758 to 3199975 tid 3 2023-06-27T12:38:16.728 INFO:tasks.rados.rados.0.smithi138.stdout:117: write oid 50 current snap is 6 2023-06-27T12:38:16.729 INFO:tasks.rados.rados.0.smithi138.stdout:117: seq_num 79 ranges {527753=708201,1856548=737690,3286534=41219} 2023-06-27T12:38:16.735 INFO:tasks.rados.rados.0.smithi138.stdout:117: writing smithi138138488-50 from 527753 to 1235954 tid 1 2023-06-27T12:38:16.743 INFO:tasks.rados.rados.0.smithi138.stdout:117: writing smithi138138488-50 from 1856548 to 2594238 tid 2 2023-06-27T12:38:16.747 INFO:tasks.rados.rados.0.smithi138.stdout:117: writing smithi138138488-50 from 3286534 to 3327753 tid 3 2023-06-27T12:38:16.748 INFO:tasks.rados.rados.0.smithi138.stdout:118: write oid 49 current snap is 6 2023-06-27T12:38:16.748 INFO:tasks.rados.rados.0.smithi138.stdout:118: seq_num 80 ranges {544560=783450,1915030=481348,2797604=479761} 2023-06-27T12:38:16.755 INFO:tasks.rados.rados.0.smithi138.stdout:118: writing smithi138138488-49 from 544560 to 1328010 tid 1 2023-06-27T12:38:16.762 INFO:tasks.rados.rados.0.smithi138.stdout:118: writing smithi138138488-49 from 1915030 to 2396378 tid 2 2023-06-27T12:38:16.768 INFO:tasks.rados.rados.0.smithi138.stdout:118: writing smithi138138488-49 from 2797604 to 3277365 tid 3 2023-06-27T12:38:16.768 INFO:tasks.rados.rados.0.smithi138.stdout:119: write (excl) oid 26 current snap is 6 2023-06-27T12:38:16.768 INFO:tasks.rados.rados.0.smithi138.stdout:116: finishing write tid 1 to smithi138138488-28 2023-06-27T12:38:16.768 INFO:tasks.rados.rados.0.smithi138.stdout:119: seq_num 81 ranges {561367=458699,1573512=741359,2941380=419987} 2023-06-27T12:38:16.774 INFO:tasks.rados.rados.0.smithi138.stdout:119: writing smithi138138488-26 from 561367 to 1020066 tid 1 2023-06-27T12:38:16.782 INFO:tasks.rados.rados.0.smithi138.stdout:119: writing smithi138138488-26 from 1573512 to 2314871 tid 2 2023-06-27T12:38:16.788 INFO:tasks.rados.rados.0.smithi138.stdout:119: writing smithi138138488-26 from 2941380 to 3361367 tid 3 2023-06-27T12:38:16.788 INFO:tasks.rados.rados.0.smithi138.stdout:120: write (excl) oid 44 current snap is 6 2023-06-27T12:38:16.788 INFO:tasks.rados.rados.0.smithi138.stdout:120: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2023-06-27T12:38:16.794 INFO:tasks.rados.rados.0.smithi138.stdout:120: writing smithi138138488-44 from 578174 to 1112122 tid 1 2023-06-27T12:38:16.799 INFO:tasks.rados.rados.0.smithi138.stdout:120: writing smithi138138488-44 from 1515641 to 2000658 tid 2 2023-06-27T12:38:16.807 INFO:tasks.rados.rados.0.smithi138.stdout:120: writing smithi138138488-44 from 2736097 to 3238402 tid 3 2023-06-27T12:38:16.807 INFO:tasks.rados.rados.0.smithi138.stdout:121: read oid 22 snap -1 2023-06-27T12:38:16.807 INFO:tasks.rados.rados.0.smithi138.stdout:121: expect (ObjNum 68 snap 3 seq_num 68) 2023-06-27T12:38:16.807 INFO:tasks.rados.rados.0.smithi138.stdout:122: snap_remove snap 5 2023-06-27T12:38:17.439 INFO:tasks.rados.rados.0.smithi138.stdout:122: done (6 left) 2023-06-27T12:38:17.440 INFO:tasks.rados.rados.0.smithi138.stdout:123: read oid 37 snap -1 2023-06-27T12:38:17.440 INFO:tasks.rados.rados.0.smithi138.stdout:123: expect (ObjNum 54 snap 0 seq_num 54) 2023-06-27T12:38:17.441 INFO:tasks.rados.rados.0.smithi138.stdout:124: snap_remove snap 4 2023-06-27T12:38:18.441 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:18.446 INFO:tasks.rados.rados.0.smithi138.stdout:124: done (7 left) 2023-06-27T12:38:18.447 INFO:tasks.rados.rados.0.smithi138.stdout:125: snap_remove snap 3 2023-06-27T12:38:18.579 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:18.574+0000 7fb4628ef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:18.578+0000 7fb4628ef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:18.836 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:18.836 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:18.837 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:18.963 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:18.957+0000 7fae090bb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:18.966 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:18.963+0000 7fae090bb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:19.161 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:19.162 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:19.174 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-06-27T12:38:18.441464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":120742093,"num_objects":60,"num_object_clones":11,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":311,"num_read_kb":99381,"num_write":419,"num_write_kb":79886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":297432,"kb_used_data":190136,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377189928,"statfs":{"total":386547056640,"available":386242486272,"internally_reserved":0,"allocated":194699264,"data_stored":198493691,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014690"},"pg_stats":[{"pgid":"3.f","version":"18'4","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.438179+0000","last_change":"2023-06-27T12:38:13.442501+0000","last_active":"2023-06-27T12:38:13.438179+0000","last_peered":"2023-06-27T12:38:13.438179+0000","last_clean":"2023-06-27T12:38:13.438179+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:13.438179+0000","last_undegraded":"2023-06-27T12:38:13.438179+0000","last_fullsized":"2023-06-27T12:38:13.438179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'21","reported_seq":55,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.570272+0000","last_change":"2023-06-27T12:38:13.640116+0000","last_active":"2023-06-27T12:38:13.570272+0000","last_peered":"2023-06-27T12:38:13.570272+0000","last_clean":"2023-06-27T12:38:13.570272+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:13.570272+0000","last_undegraded":"2023-06-27T12:38:13.570272+0000","last_fullsized":"2023-06-27T12:38:13.570272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8285752,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10689,"num_write":20,"num_write_kb":5702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"18'23","reported_seq":59,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.466407+0000","last_change":"2023-06-27T12:38:13.526147+0000","last_active":"2023-06-27T12:38:13.466407+0000","last_peered":"2023-06-27T12:38:13.466407+0000","last_clean":"2023-06-27T12:38:13.466407+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:13.466407+0000","last_undegraded":"2023-06-27T12:38:13.466407+0000","last_fullsized":"2023-06-27T12:38:13.466407+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12773155,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":4857,"num_write":30,"num_write_kb":8165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"20'14","reported_seq":44,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.440067+0000","last_change":"2023-06-27T12:38:13.481579+0000","last_active":"2023-06-27T12:38:13.440067+0000","last_peered":"2023-06-27T12:38:13.440067+0000","last_clean":"2023-06-27T12:38:13.440067+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:13.440067+0000","last_undegraded":"2023-06-27T12:38:13.440067+0000","last_fullsized":"2023-06-27T12:38:13.440067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8235331,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":17,"num_write_kb":4732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"18'6","reported_seq":40,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.533934+0000","last_change":"2023-06-27T12:38:13.503247+0000","last_active":"2023-06-27T12:38:13.533934+0000","last_peered":"2023-06-27T12:38:13.533934+0000","last_clean":"2023-06-27T12:38:13.533934+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:13.533934+0000","last_undegraded":"2023-06-27T12:38:13.533934+0000","last_fullsized":"2023-06-27T12:38:13.533934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4352947,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":11423,"num_write":8,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"24'27","reported_seq":60,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.437838+0000","last_change":"2023-06-27T12:38:13.581239+0000","last_active":"2023-06-27T12:38:13.437838+0000","last_peered":"2023-06-27T12:38:13.437838+0000","last_clean":"2023-06-27T12:38:13.437838+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:13.437838+0000","last_undegraded":"2023-06-27T12:38:13.437838+0000","last_fullsized":"2023-06-27T12:38:13.437838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13596698,"num_objects":7,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":5285,"num_write":30,"num_write_kb":7976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"23'16","reported_seq":52,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.442038+0000","last_change":"2023-06-27T12:38:13.631309+0000","last_active":"2023-06-27T12:38:13.442038+0000","last_peered":"2023-06-27T12:38:13.442038+0000","last_clean":"2023-06-27T12:38:13.442038+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:13.442038+0000","last_undegraded":"2023-06-27T12:38:13.442038+0000","last_fullsized":"2023-06-27T12:38:13.442038+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731121,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":8972,"num_write":20,"num_write_kb":5591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"18'3","reported_seq":36,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.562661+0000","last_change":"2023-06-27T12:38:13.472168+0000","last_active":"2023-06-27T12:38:13.562661+0000","last_peered":"2023-06-27T12:38:13.562661+0000","last_clean":"2023-06-27T12:38:13.562661+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:13.562661+0000","last_undegraded":"2023-06-27T12:38:13.562661+0000","last_fullsized":"2023-06-27T12:38:13.562661+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2823543,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":11709,"num_write":4,"num_write_kb":1548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.466669+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:13.466669+0000","last_peered":"2023-06-27T12:38:13.466669+0000","last_clean":"2023-06-27T12:38:13.466669+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:13.466669+0000","last_undegraded":"2023-06-27T12:38:13.466669+0000","last_fullsized":"2023-06-27T12:38:13.466669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.494250+0000","last_change":"2023-06-27T12:38:13.536173+0000","last_active":"2023-06-27T12:38:13.494250+0000","last_peered":"2023-06-27T12:38:13.494250+0000","last_clean":"2023-06-27T12:38:13.494250+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:13.494250+0000","last_undegraded":"2023-06-27T12:38:13.494250+0000","last_fullsized":"2023-06-27T12:38:13.494250+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.434341+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:13.434341+0000","last_peered":"2023-06-27T12:38:13.434341+0000","last_clean":"2023-06-27T12:38:13.434341+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:13.434341+0000","last_undegraded":"2023-06-27T12:38:13.434341+0000","last_fullsized":"2023-06-27T12:38:13.434341+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'19","reported_seq":51,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.467489+0000","last_change":"2023-06-27T12:38:13.625242+0000","last_active":"2023-06-27T12:38:13.467489+0000","last_peered":"2023-06-27T12:38:13.467489+0000","last_clean":"2023-06-27T12:38:13.467489+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:13.467489+0000","last_undegraded":"2023-06-27T12:38:13.467489+0000","last_fullsized":"2023-06-27T12:38:13.467489+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6151296,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":21,"num_write_kb":5953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.479956+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:13.479956+0000","last_peered":"2023-06-27T12:38:13.479956+0000","last_clean":"2023-06-27T12:38:13.479956+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:13.479956+0000","last_undegraded":"2023-06-27T12:38:13.479956+0000","last_fullsized":"2023-06-27T12:38:13.479956+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'33","reported_seq":71,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.558813+0000","last_change":"2023-06-27T12:38:13.591975+0000","last_active":"2023-06-27T12:38:13.558813+0000","last_peered":"2023-06-27T12:38:13.558813+0000","last_clean":"2023-06-27T12:38:13.558813+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:13.558813+0000","last_undegraded":"2023-06-27T12:38:13.558813+0000","last_fullsized":"2023-06-27T12:38:13.558813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13680733,"num_objects":7,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":10334,"num_write":38,"num_write_kb":11192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.458523+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:13.458523+0000","last_peered":"2023-06-27T12:38:13.458523+0000","last_clean":"2023-06-27T12:38:13.458523+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:13.458523+0000","last_undegraded":"2023-06-27T12:38:13.458523+0000","last_fullsized":"2023-06-27T12:38:13.458523+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":158,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.464204+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:13.464204+0000","last_peered":"2023-06-27T12:38:13.464204+0000","last_clean":"2023-06-27T12:38:13.464204+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:13.464204+0000","last_undegraded":"2023-06-27T12:38:13.464204+0000","last_fullsized":"2023-06-27T12:38:13.464204+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"23'20","reported_seq":51,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.470461+0000","last_change":"2023-06-27T12:38:13.601369+0000","last_active":"2023-06-27T12:38:13.470461+0000","last_peered":"2023-06-27T12:38:13.470461+0000","last_clean":"2023-06-27T12:38:13.470461+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:13.470461+0000","last_undegraded":"2023-06-27T12:38:13.470461+0000","last_fullsized":"2023-06-27T12:38:13.470461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8789929,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":13,"num_read_kb":6,"num_write":24,"num_write_kb":5604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.432334+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:13.432334+0000","last_peered":"2023-06-27T12:38:13.432334+0000","last_clean":"2023-06-27T12:38:13.432334+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:13.432334+0000","last_undegraded":"2023-06-27T12:38:13.432334+0000","last_fullsized":"2023-06-27T12:38:13.432334+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"22'16","reported_seq":55,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.523837+0000","last_change":"2023-06-27T12:38:13.480231+0000","last_active":"2023-06-27T12:38:13.523837+0000","last_peered":"2023-06-27T12:38:13.523837+0000","last_clean":"2023-06-27T12:38:13.523837+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:13.523837+0000","last_undegraded":"2023-06-27T12:38:13.523837+0000","last_fullsized":"2023-06-27T12:38:13.523837+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9848770,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":18623,"num_write":20,"num_write_kb":5449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.470646+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:13.470646+0000","last_peered":"2023-06-27T12:38:13.470646+0000","last_clean":"2023-06-27T12:38:13.470646+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:13.470646+0000","last_undegraded":"2023-06-27T12:38:13.470646+0000","last_fullsized":"2023-06-27T12:38:13.470646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"19'12","reported_seq":50,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.558159+0000","last_change":"2023-06-27T12:38:13.471242+0000","last_active":"2023-06-27T12:38:13.558159+0000","last_peered":"2023-06-27T12:38:13.558159+0000","last_clean":"2023-06-27T12:38:13.558159+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:13.558159+0000","last_undegraded":"2023-06-27T12:38:13.558159+0000","last_fullsized":"2023-06-27T12:38:13.558159+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9831963,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":9844,"num_write":16,"num_write_kb":4475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"18'6","reported_seq":36,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.460811+0000","last_change":"2023-06-27T12:38:13.479437+0000","last_active":"2023-06-27T12:38:13.460811+0000","last_peered":"2023-06-27T12:38:13.460811+0000","last_clean":"2023-06-27T12:38:13.460811+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:13.460811+0000","last_undegraded":"2023-06-27T12:38:13.460811+0000","last_fullsized":"2023-06-27T12:38:13.460811+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4873964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3465,"num_write":8,"num_write_kb":2305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.459567+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:13.459567+0000","last_peered":"2023-06-27T12:38:13.459567+0000","last_clean":"2023-06-27T12:38:13.459567+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:13.459567+0000","last_undegraded":"2023-06-27T12:38:13.459567+0000","last_fullsized":"2023-06-27T12:38:13.459567+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.460382+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:13.460382+0000","last_peered":"2023-06-27T12:38:13.460382+0000","last_clean":"2023-06-27T12:38:13.460382+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:13.460382+0000","last_undegraded":"2023-06-27T12:38:13.460382+0000","last_fullsized":"2023-06-27T12:38:13.460382+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"25'25","reported_seq":55,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-27T12:38:13.641350+0000","last_change":"2023-06-27T12:38:13.642733+0000","last_active":"2023-06-27T12:38:13.641350+0000","last_peered":"2023-06-27T12:38:13.641350+0000","last_clean":"2023-06-27T12:38:13.641350+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:13.641350+0000","last_undegraded":"2023-06-27T12:38:13.641350+0000","last_fullsized":"2023-06-27T12:38:13.641350+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9176523,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":4073,"num_write":22,"num_write_kb":6359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":120151725,"num_objects":58,"num_object_clones":11,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":208,"num_read_kb":99290,"num_write":282,"num_write_kb":77992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":245,"ondisk_log_size":245,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78456,"kb_used_data":51632,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94293384,"statfs":{"total":96636764160,"available":96556425216,"internally_reserved":0,"allocated":52871168,"data_stored":53698050,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":71412,"kb_used_data":44588,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94300428,"statfs":{"total":96636764160,"available":96563638272,"internally_reserved":0,"allocated":45658112,"data_stored":46398685,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705685,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":70128,"kb_used_data":43304,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94301712,"statfs":{"total":96636764160,"available":96564953088,"internally_reserved":0,"allocated":44343296,"data_stored":45707994,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77436,"kb_used_data":50612,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94294404,"statfs":{"total":96636764160,"available":96557469696,"internally_reserved":0,"allocated":51826688,"data_stored":52688962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:19.447 INFO:tasks.rados.rados.0.smithi138.stdout:125: done (7 left) 2023-06-27T12:38:19.447 INFO:tasks.rados.rados.0.smithi138.stdout:126: write oid 36 current snap is 6 2023-06-27T12:38:19.447 INFO:tasks.rados.rados.0.smithi138.stdout:116: finishing write tid 2 to smithi138138488-28 2023-06-27T12:38:19.447 INFO:tasks.rados.rados.0.smithi138.stdout:126: seq_num 83 ranges {594981=609197,1974123=745028,3279873=115108} 2023-06-27T12:38:19.453 INFO:tasks.rados.rados.0.smithi138.stdout:126: writing smithi138138488-36 from 594981 to 1204178 tid 1 2023-06-27T12:38:19.462 INFO:tasks.rados.rados.0.smithi138.stdout:126: writing smithi138138488-36 from 1974123 to 2719151 tid 2 2023-06-27T12:38:19.466 INFO:tasks.rados.rados.0.smithi138.stdout:126: writing smithi138138488-36 from 3279873 to 3394981 tid 3 2023-06-27T12:38:19.466 INFO:tasks.rados.rados.0.smithi138.stdout:127: write (excl) oid 35 current snap is 6 2023-06-27T12:38:19.466 INFO:tasks.rados.rados.0.smithi138.stdout:127: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2023-06-27T12:38:19.471 INFO:tasks.rados.rados.0.smithi138.stdout:127: writing smithi138138488-35 from 611788 to 1012587 tid 1 2023-06-27T12:38:19.478 INFO:tasks.rados.rados.0.smithi138.stdout:127: writing smithi138138488-35 from 1748958 to 2237644 tid 2 2023-06-27T12:38:19.485 INFO:tasks.rados.rados.0.smithi138.stdout:127: writing smithi138138488-35 from 2907296 to 3411788 tid 3 2023-06-27T12:38:19.486 INFO:tasks.rados.rados.0.smithi138.stdout:128: write oid 2 current snap is 6 2023-06-27T12:38:19.486 INFO:tasks.rados.rados.0.smithi138.stdout:128: seq_num 85 ranges {628595=476048,1807440=632344,2934719=452474} 2023-06-27T12:38:19.492 INFO:tasks.rados.rados.0.smithi138.stdout:128: writing smithi138138488-2 from 628595 to 1104643 tid 1 2023-06-27T12:38:19.499 INFO:tasks.rados.rados.0.smithi138.stdout:128: writing smithi138138488-2 from 1807440 to 2439784 tid 2 2023-06-27T12:38:19.505 INFO:tasks.rados.rados.0.smithi138.stdout:128: writing smithi138138488-2 from 2934719 to 3387193 tid 3 2023-06-27T12:38:19.506 INFO:tasks.rados.rados.0.smithi138.stdout:129: read oid 9 snap -1 2023-06-27T12:38:19.506 INFO:tasks.rados.rados.0.smithi138.stdout:129: expect (ObjNum 8 snap 0 seq_num 8) 2023-06-27T12:38:19.506 INFO:tasks.rados.rados.0.smithi138.stdout:130: read oid 46 snap -1 2023-06-27T12:38:19.506 INFO:tasks.rados.rados.0.smithi138.stdout:130: expect (ObjNum 62 snap 2 seq_num 62) 2023-06-27T12:38:19.506 INFO:tasks.rados.rados.0.smithi138.stdout:131: write oid 1 current snap is 6 2023-06-27T12:38:19.506 INFO:tasks.rados.rados.0.smithi138.stdout:117: finishing write tid 1 to smithi138138488-50 2023-06-27T12:38:19.506 INFO:tasks.rados.rados.0.smithi138.stdout:116: finishing write tid 3 to smithi138138488-28 2023-06-27T12:38:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:116: finishing write tid 4 to smithi138138488-28 2023-06-27T12:38:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:116: finishing write tid 5 to smithi138138488-28 2023-06-27T12:38:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:116: oid 28 updating version 0 to 25 2023-06-27T12:38:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:116: oid 28 version 25 is already newer than 25 2023-06-27T12:38:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:116: oid 28 version 25 is already newer than 25 2023-06-27T12:38:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:116: oid 28 version 25 is already newer than 23 2023-06-27T12:38:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 25 (ObjNum 78 snap 6 seq_num 78) dirty exists 2023-06-27T12:38:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:116: left oid 28 (ObjNum 78 snap 6 seq_num 78) 2023-06-27T12:38:19.508 INFO:tasks.rados.rados.0.smithi138.stdout:117: finishing write tid 2 to smithi138138488-50 2023-06-27T12:38:19.508 INFO:tasks.rados.rados.0.smithi138.stdout:117: finishing write tid 3 to smithi138138488-50 2023-06-27T12:38:19.508 INFO:tasks.rados.rados.0.smithi138.stdout:131: seq_num 86 ranges {645402=551297,1749569=492355,2845789=599613} 2023-06-27T12:38:19.512 INFO:tasks.rados.rados.0.smithi138.stdout:131: writing smithi138138488-1 from 645402 to 1196699 tid 1 2023-06-27T12:38:19.517 INFO:tasks.rados.rados.0.smithi138.stdout:131: writing smithi138138488-1 from 1749569 to 2241924 tid 2 2023-06-27T12:38:19.525 INFO:tasks.rados.rados.0.smithi138.stdout:131: writing smithi138138488-1 from 2845789 to 3445402 tid 3 2023-06-27T12:38:19.525 INFO:tasks.rados.rados.0.smithi138.stdout:116: done (12 left) 2023-06-27T12:38:19.525 INFO:tasks.rados.rados.0.smithi138.stdout:132: read oid 20 snap -1 2023-06-27T12:38:19.525 INFO:tasks.rados.rados.0.smithi138.stdout:132: expect (ObjNum 19 snap 0 seq_num 19) 2023-06-27T12:38:19.525 INFO:tasks.rados.rados.0.smithi138.stdout:133: write oid 28 current snap is 6 2023-06-27T12:38:19.526 INFO:tasks.rados.rados.0.smithi138.stdout:117: finishing write tid 4 to smithi138138488-50 2023-06-27T12:38:19.526 INFO:tasks.rados.rados.0.smithi138.stdout:117: finishing write tid 5 to smithi138138488-50 2023-06-27T12:38:19.526 INFO:tasks.rados.rados.0.smithi138.stdout:117: oid 50 updating version 0 to 8 2023-06-27T12:38:19.526 INFO:tasks.rados.rados.0.smithi138.stdout:117: oid 50 version 8 is already newer than 8 2023-06-27T12:38:19.526 INFO:tasks.rados.rados.0.smithi138.stdout:117: oid 50 version 8 is already newer than 8 2023-06-27T12:38:19.526 INFO:tasks.rados.rados.0.smithi138.stdout:117: oid 50 version 8 is already newer than 8 2023-06-27T12:38:19.526 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 8 (ObjNum 79 snap 6 seq_num 79) dirty exists 2023-06-27T12:38:19.527 INFO:tasks.rados.rados.0.smithi138.stdout:117: left oid 50 (ObjNum 79 snap 6 seq_num 79) 2023-06-27T12:38:19.527 INFO:tasks.rados.rados.0.smithi138.stdout:118: finishing write tid 1 to smithi138138488-49 2023-06-27T12:38:19.527 INFO:tasks.rados.rados.0.smithi138.stdout:120: finishing write tid 1 to smithi138138488-44 2023-06-27T12:38:19.527 INFO:tasks.rados.rados.0.smithi138.stdout:118: finishing write tid 2 to smithi138138488-49 2023-06-27T12:38:19.527 INFO:tasks.rados.rados.0.smithi138.stdout:133: seq_num 87 ranges {662209=626546,1808051=636013,2873212=588997} 2023-06-27T12:38:19.532 INFO:tasks.rados.rados.0.smithi138.stdout:133: writing smithi138138488-28 from 662209 to 1288755 tid 1 2023-06-27T12:38:19.539 INFO:tasks.rados.rados.0.smithi138.stdout:133: writing smithi138138488-28 from 1808051 to 2444064 tid 2 2023-06-27T12:38:19.546 INFO:tasks.rados.rados.0.smithi138.stdout:133: writing smithi138138488-28 from 2873212 to 3462209 tid 3 2023-06-27T12:38:19.546 INFO:tasks.rados.rados.0.smithi138.stdout:117: done (13 left) 2023-06-27T12:38:19.546 INFO:tasks.rados.rados.0.smithi138.stdout:134: snap_create 2023-06-27T12:38:19.547 INFO:tasks.rados.rados.0.smithi138.stdout:118: finishing write tid 3 to smithi138138488-49 2023-06-27T12:38:19.547 INFO:tasks.rados.rados.0.smithi138.stdout:118: finishing write tid 4 to smithi138138488-49 2023-06-27T12:38:19.547 INFO:tasks.rados.rados.0.smithi138.stdout:119: finishing write tid 1 to smithi138138488-26 2023-06-27T12:38:19.547 INFO:tasks.rados.rados.0.smithi138.stdout:118: finishing write tid 5 to smithi138138488-49 2023-06-27T12:38:19.547 INFO:tasks.rados.rados.0.smithi138.stdout:118: oid 49 updating version 0 to 30 2023-06-27T12:38:19.547 INFO:tasks.rados.rados.0.smithi138.stdout:118: oid 49 version 30 is already newer than 30 2023-06-27T12:38:19.547 INFO:tasks.rados.rados.0.smithi138.stdout:118: oid 49 version 30 is already newer than 30 2023-06-27T12:38:19.548 INFO:tasks.rados.rados.0.smithi138.stdout:118: oid 49 version 30 is already newer than 30 2023-06-27T12:38:19.548 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 49 v 30 (ObjNum 80 snap 6 seq_num 80) dirty exists 2023-06-27T12:38:19.548 INFO:tasks.rados.rados.0.smithi138.stdout:118: left oid 49 (ObjNum 80 snap 6 seq_num 80) 2023-06-27T12:38:19.548 INFO:tasks.rados.rados.0.smithi138.stdout:120: finishing write tid 2 to smithi138138488-44 2023-06-27T12:38:19.549 INFO:tasks.rados.rados.0.smithi138.stdout:121: expect (ObjNum 68 snap 3 seq_num 68) 2023-06-27T12:38:19.774 INFO:tasks.rados.rados.0.smithi138.stdout:119: finishing write tid 2 to smithi138138488-26 2023-06-27T12:38:19.774 INFO:tasks.rados.rados.0.smithi138.stdout:119: finishing write tid 3 to smithi138138488-26 2023-06-27T12:38:19.774 INFO:tasks.rados.rados.0.smithi138.stdout:119: finishing write tid 4 to smithi138138488-26 2023-06-27T12:38:19.775 INFO:tasks.rados.rados.0.smithi138.stdout:119: finishing write tid 5 to smithi138138488-26 2023-06-27T12:38:19.775 INFO:tasks.rados.rados.0.smithi138.stdout:119: oid 26 updating version 0 to 18 2023-06-27T12:38:19.775 INFO:tasks.rados.rados.0.smithi138.stdout:119: oid 26 version 18 is already newer than 18 2023-06-27T12:38:19.775 INFO:tasks.rados.rados.0.smithi138.stdout:119: oid 26 version 18 is already newer than 15 2023-06-27T12:38:19.775 INFO:tasks.rados.rados.0.smithi138.stdout:119: oid 26 version 18 is already newer than 18 2023-06-27T12:38:19.776 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 18 (ObjNum 81 snap 6 seq_num 81) dirty exists 2023-06-27T12:38:19.776 INFO:tasks.rados.rados.0.smithi138.stdout:119: left oid 26 (ObjNum 81 snap 6 seq_num 81) 2023-06-27T12:38:19.776 INFO:tasks.rados.rados.0.smithi138.stdout:120: finishing write tid 3 to smithi138138488-44 2023-06-27T12:38:19.776 INFO:tasks.rados.rados.0.smithi138.stdout:120: finishing write tid 4 to smithi138138488-44 2023-06-27T12:38:19.776 INFO:tasks.rados.rados.0.smithi138.stdout:120: finishing write tid 5 to smithi138138488-44 2023-06-27T12:38:19.776 INFO:tasks.rados.rados.0.smithi138.stdout:120: oid 44 updating version 0 to 34 2023-06-27T12:38:19.776 INFO:tasks.rados.rados.0.smithi138.stdout:120: oid 44 version 34 is already newer than 30 2023-06-27T12:38:19.777 INFO:tasks.rados.rados.0.smithi138.stdout:120: oid 44 version 34 is already newer than 34 2023-06-27T12:38:19.777 INFO:tasks.rados.rados.0.smithi138.stdout:120: oid 44 version 34 is already newer than 34 2023-06-27T12:38:19.777 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 34 (ObjNum 82 snap 6 seq_num 82) dirty exists 2023-06-27T12:38:19.777 INFO:tasks.rados.rados.0.smithi138.stdout:120: left oid 44 (ObjNum 82 snap 6 seq_num 82) 2023-06-27T12:38:19.777 INFO:tasks.rados.rados.0.smithi138.stdout:123: expect (ObjNum 54 snap 0 seq_num 54) 2023-06-27T12:38:19.920 INFO:tasks.rados.rados.0.smithi138.stdout:127: finishing write tid 1 to smithi138138488-35 2023-06-27T12:38:19.921 INFO:tasks.rados.rados.0.smithi138.stdout:126: finishing write tid 1 to smithi138138488-36 2023-06-27T12:38:19.921 INFO:tasks.rados.rados.0.smithi138.stdout:129: expect (ObjNum 8 snap 0 seq_num 8) 2023-06-27T12:38:20.000 INFO:tasks.rados.rados.0.smithi138.stdout:127: finishing write tid 2 to smithi138138488-35 2023-06-27T12:38:20.000 INFO:tasks.rados.rados.0.smithi138.stdout:130: expect (ObjNum 62 snap 2 seq_num 62) 2023-06-27T12:38:20.113 INFO:tasks.rados.rados.0.smithi138.stdout:126: finishing write tid 2 to smithi138138488-36 2023-06-27T12:38:20.114 INFO:tasks.rados.rados.0.smithi138.stdout:126: finishing write tid 3 to smithi138138488-36 2023-06-27T12:38:20.114 INFO:tasks.rados.rados.0.smithi138.stdout:127: finishing write tid 3 to smithi138138488-35 2023-06-27T12:38:20.114 INFO:tasks.rados.rados.0.smithi138.stdout:127: finishing write tid 4 to smithi138138488-35 2023-06-27T12:38:20.114 INFO:tasks.rados.rados.0.smithi138.stdout:127: finishing write tid 5 to smithi138138488-35 2023-06-27T12:38:20.114 INFO:tasks.rados.rados.0.smithi138.stdout:127: oid 35 updating version 0 to 39 2023-06-27T12:38:20.114 INFO:tasks.rados.rados.0.smithi138.stdout:127: oid 35 version 39 is already newer than 39 2023-06-27T12:38:20.114 INFO:tasks.rados.rados.0.smithi138.stdout:127: oid 35 version 39 is already newer than 36 2023-06-27T12:38:20.114 INFO:tasks.rados.rados.0.smithi138.stdout:127: oid 35 version 39 is already newer than 39 2023-06-27T12:38:20.115 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 39 (ObjNum 84 snap 6 seq_num 84) dirty exists 2023-06-27T12:38:20.115 INFO:tasks.rados.rados.0.smithi138.stdout:127: left oid 35 (ObjNum 84 snap 6 seq_num 84) 2023-06-27T12:38:20.115 INFO:tasks.rados.rados.0.smithi138.stdout:126: finishing write tid 4 to smithi138138488-36 2023-06-27T12:38:20.115 INFO:tasks.rados.rados.0.smithi138.stdout:126: finishing write tid 5 to smithi138138488-36 2023-06-27T12:38:20.115 INFO:tasks.rados.rados.0.smithi138.stdout:126: oid 36 updating version 0 to 27 2023-06-27T12:38:20.115 INFO:tasks.rados.rados.0.smithi138.stdout:126: oid 36 version 27 is already newer than 27 2023-06-27T12:38:20.115 INFO:tasks.rados.rados.0.smithi138.stdout:126: oid 36 version 27 is already newer than 27 2023-06-27T12:38:20.116 INFO:tasks.rados.rados.0.smithi138.stdout:126: oid 36 version 27 is already newer than 27 2023-06-27T12:38:20.116 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 27 (ObjNum 83 snap 6 seq_num 83) dirty exists 2023-06-27T12:38:20.116 INFO:tasks.rados.rados.0.smithi138.stdout:126: left oid 36 (ObjNum 83 snap 6 seq_num 83) 2023-06-27T12:38:20.116 INFO:tasks.rados.rados.0.smithi138.stdout:132: expect (ObjNum 19 snap 0 seq_num 19) 2023-06-27T12:38:20.201 INFO:tasks.rados.rados.0.smithi138.stdout:131: finishing write tid 1 to smithi138138488-1 2023-06-27T12:38:20.202 INFO:tasks.rados.rados.0.smithi138.stdout:133: finishing write tid 1 to smithi138138488-28 2023-06-27T12:38:20.202 INFO:tasks.rados.rados.0.smithi138.stdout:128: finishing write tid 1 to smithi138138488-2 2023-06-27T12:38:20.202 INFO:tasks.rados.rados.0.smithi138.stdout:128: finishing write tid 2 to smithi138138488-2 2023-06-27T12:38:20.203 INFO:tasks.rados.rados.0.smithi138.stdout:131: finishing write tid 2 to smithi138138488-1 2023-06-27T12:38:20.203 INFO:tasks.rados.rados.0.smithi138.stdout:131: finishing write tid 3 to smithi138138488-1 2023-06-27T12:38:20.203 INFO:tasks.rados.rados.0.smithi138.stdout:131: finishing write tid 4 to smithi138138488-1 2023-06-27T12:38:20.203 INFO:tasks.rados.rados.0.smithi138.stdout:131: finishing write tid 5 to smithi138138488-1 2023-06-27T12:38:20.204 INFO:tasks.rados.rados.0.smithi138.stdout:131: oid 1 updating version 0 to 28 2023-06-27T12:38:20.204 INFO:tasks.rados.rados.0.smithi138.stdout:131: oid 1 version 28 is already newer than 28 2023-06-27T12:38:20.204 INFO:tasks.rados.rados.0.smithi138.stdout:131: oid 1 version 28 is already newer than 28 2023-06-27T12:38:20.205 INFO:tasks.rados.rados.0.smithi138.stdout:131: oid 1 version 28 is already newer than 28 2023-06-27T12:38:20.205 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 28 (ObjNum 86 snap 6 seq_num 86) dirty exists 2023-06-27T12:38:20.205 INFO:tasks.rados.rados.0.smithi138.stdout:131: left oid 1 (ObjNum 86 snap 6 seq_num 86) 2023-06-27T12:38:20.205 INFO:tasks.rados.rados.0.smithi138.stdout:133: finishing write tid 2 to smithi138138488-28 2023-06-27T12:38:20.206 INFO:tasks.rados.rados.0.smithi138.stdout:133: finishing write tid 3 to smithi138138488-28 2023-06-27T12:38:20.206 INFO:tasks.rados.rados.0.smithi138.stdout:133: finishing write tid 4 to smithi138138488-28 2023-06-27T12:38:20.206 INFO:tasks.rados.rados.0.smithi138.stdout:133: finishing write tid 5 to smithi138138488-28 2023-06-27T12:38:20.207 INFO:tasks.rados.rados.0.smithi138.stdout:133: oid 28 updating version 0 to 31 2023-06-27T12:38:20.207 INFO:tasks.rados.rados.0.smithi138.stdout:133: oid 28 version 31 is already newer than 31 2023-06-27T12:38:20.207 INFO:tasks.rados.rados.0.smithi138.stdout:133: oid 28 version 31 is already newer than 31 2023-06-27T12:38:20.207 INFO:tasks.rados.rados.0.smithi138.stdout:133: oid 28 version 31 is already newer than 31 2023-06-27T12:38:20.208 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 31 (ObjNum 87 snap 6 seq_num 87) dirty exists 2023-06-27T12:38:20.208 INFO:tasks.rados.rados.0.smithi138.stdout:133: left oid 28 (ObjNum 87 snap 6 seq_num 87) 2023-06-27T12:38:20.208 INFO:tasks.rados.rados.0.smithi138.stdout:128: finishing write tid 3 to smithi138138488-2 2023-06-27T12:38:20.209 INFO:tasks.rados.rados.0.smithi138.stdout:128: finishing write tid 4 to smithi138138488-2 2023-06-27T12:38:20.209 INFO:tasks.rados.rados.0.smithi138.stdout:128: finishing write tid 5 to smithi138138488-2 2023-06-27T12:38:20.209 INFO:tasks.rados.rados.0.smithi138.stdout:128: oid 2 updating version 0 to 43 2023-06-27T12:38:20.209 INFO:tasks.rados.rados.0.smithi138.stdout:128: oid 2 version 43 is already newer than 43 2023-06-27T12:38:20.210 INFO:tasks.rados.rados.0.smithi138.stdout:128: oid 2 version 43 is already newer than 43 2023-06-27T12:38:20.210 INFO:tasks.rados.rados.0.smithi138.stdout:128: oid 2 version 43 is already newer than 43 2023-06-27T12:38:20.210 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 43 (ObjNum 85 snap 6 seq_num 85) dirty exists 2023-06-27T12:38:20.210 INFO:tasks.rados.rados.0.smithi138.stdout:128: left oid 2 (ObjNum 85 snap 6 seq_num 85) 2023-06-27T12:38:20.454 INFO:tasks.rados.rados.0.smithi138.stdout:118: done (13 left) 2023-06-27T12:38:20.454 INFO:tasks.rados.rados.0.smithi138.stdout:119: done (12 left) 2023-06-27T12:38:20.454 INFO:tasks.rados.rados.0.smithi138.stdout:120: done (11 left) 2023-06-27T12:38:20.454 INFO:tasks.rados.rados.0.smithi138.stdout:121: done (10 left) 2023-06-27T12:38:20.454 INFO:tasks.rados.rados.0.smithi138.stdout:123: done (9 left) 2023-06-27T12:38:20.455 INFO:tasks.rados.rados.0.smithi138.stdout:126: done (8 left) 2023-06-27T12:38:20.455 INFO:tasks.rados.rados.0.smithi138.stdout:127: done (7 left) 2023-06-27T12:38:20.455 INFO:tasks.rados.rados.0.smithi138.stdout:128: done (6 left) 2023-06-27T12:38:20.455 INFO:tasks.rados.rados.0.smithi138.stdout:129: done (5 left) 2023-06-27T12:38:20.455 INFO:tasks.rados.rados.0.smithi138.stdout:130: done (4 left) 2023-06-27T12:38:20.455 INFO:tasks.rados.rados.0.smithi138.stdout:131: done (3 left) 2023-06-27T12:38:20.455 INFO:tasks.rados.rados.0.smithi138.stdout:132: done (2 left) 2023-06-27T12:38:20.455 INFO:tasks.rados.rados.0.smithi138.stdout:133: done (1 left) 2023-06-27T12:38:20.456 INFO:tasks.rados.rados.0.smithi138.stdout:134: done (0 left) 2023-06-27T12:38:20.456 INFO:tasks.rados.rados.0.smithi138.stdout:135: write (excl) oid 42 current snap is 7 2023-06-27T12:38:20.456 INFO:tasks.rados.rados.0.smithi138.stdout:135: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2023-06-27T12:38:20.461 INFO:tasks.rados.rados.0.smithi138.stdout:135: writing smithi138138488-42 from 679016 to 1380811 tid 1 2023-06-27T12:38:20.467 INFO:tasks.rados.rados.0.smithi138.stdout:135: writing smithi138138488-42 from 1866533 to 2362557 tid 2 2023-06-27T12:38:20.472 INFO:tasks.rados.rados.0.smithi138.stdout:135: writing smithi138138488-42 from 2900635 to 3303278 tid 3 2023-06-27T12:38:20.473 INFO:tasks.rados.rados.0.smithi138.stdout:136: snap_remove snap 6 2023-06-27T12:38:20.653 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:20.779 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:20.774+0000 7f485aad7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:20.782 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:20.778+0000 7f485aad7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:21.458 INFO:tasks.rados.rados.0.smithi138.stdout:136: done (1 left) 2023-06-27T12:38:21.458 INFO:tasks.rados.rados.0.smithi138.stdout:137: read oid 2 snap -1 2023-06-27T12:38:21.458 INFO:tasks.rados.rados.0.smithi138.stdout:137: expect (ObjNum 85 snap 6 seq_num 85) 2023-06-27T12:38:21.458 INFO:tasks.rados.rados.0.smithi138.stdout:138: read oid 16 snap -1 2023-06-27T12:38:21.459 INFO:tasks.rados.rados.0.smithi138.stdout:138: expect (ObjNum 15 snap 0 seq_num 15) 2023-06-27T12:38:21.459 INFO:tasks.rados.rados.0.smithi138.stdout:139: read oid 32 snap -1 2023-06-27T12:38:21.459 INFO:tasks.rados.rados.0.smithi138.stdout:139: expect (ObjNum 31 snap 0 seq_num 31) 2023-06-27T12:38:21.459 INFO:tasks.rados.rados.0.smithi138.stdout:140: read oid 22 snap -1 2023-06-27T12:38:21.459 INFO:tasks.rados.rados.0.smithi138.stdout:140: expect (ObjNum 68 snap 3 seq_num 68) 2023-06-27T12:38:21.459 INFO:tasks.rados.rados.0.smithi138.stdout:141: write oid 17 current snap is 7 2023-06-27T12:38:21.459 INFO:tasks.rados.rados.0.smithi138.stdout:135: finishing write tid 1 to smithi138138488-42 2023-06-27T12:38:21.459 INFO:tasks.rados.rados.0.smithi138.stdout:135: finishing write tid 2 to smithi138138488-42 2023-06-27T12:38:21.460 INFO:tasks.rados.rados.0.smithi138.stdout:135: finishing write tid 3 to smithi138138488-42 2023-06-27T12:38:21.460 INFO:tasks.rados.rados.0.smithi138.stdout:135: finishing write tid 4 to smithi138138488-42 2023-06-27T12:38:21.460 INFO:tasks.rados.rados.0.smithi138.stdout:135: finishing write tid 5 to smithi138138488-42 2023-06-27T12:38:21.460 INFO:tasks.rados.rados.0.smithi138.stdout:135: oid 42 updating version 0 to 22 2023-06-27T12:38:21.460 INFO:tasks.rados.rados.0.smithi138.stdout:135: oid 42 version 22 is already newer than 22 2023-06-27T12:38:21.460 INFO:tasks.rados.rados.0.smithi138.stdout:135: oid 42 version 22 is already newer than 19 2023-06-27T12:38:21.460 INFO:tasks.rados.rados.0.smithi138.stdout:135: oid 42 version 22 is already newer than 22 2023-06-27T12:38:21.461 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 22 (ObjNum 88 snap 7 seq_num 88) dirty exists 2023-06-27T12:38:21.461 INFO:tasks.rados.rados.0.smithi138.stdout:135: left oid 42 (ObjNum 88 snap 7 seq_num 88) 2023-06-27T12:38:21.461 INFO:tasks.rados.rados.0.smithi138.stdout:141: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2023-06-27T12:38:21.466 INFO:tasks.rados.rados.0.smithi138.stdout:141: writing smithi138138488-17 from 695823 to 1472867 tid 1 2023-06-27T12:38:21.472 INFO:tasks.rados.rados.0.smithi138.stdout:141: writing smithi138138488-17 from 1925015 to 2564697 tid 2 2023-06-27T12:38:21.477 INFO:tasks.rados.rados.0.smithi138.stdout:141: writing smithi138138488-17 from 3328058 to 3495823 tid 3 2023-06-27T12:38:21.477 INFO:tasks.rados.rados.0.smithi138.stdout:135: done (5 left) 2023-06-27T12:38:21.478 INFO:tasks.rados.rados.0.smithi138.stdout:142: write (excl) oid 29 current snap is 7 2023-06-27T12:38:21.478 INFO:tasks.rados.rados.0.smithi138.stdout:142: seq_num 90 ranges {712630=452293,1583497=499693,2555481=425187} 2023-06-27T12:38:21.484 INFO:tasks.rados.rados.0.smithi138.stdout:142: writing smithi138138488-29 from 712630 to 1164923 tid 1 2023-06-27T12:38:21.489 INFO:tasks.rados.rados.0.smithi138.stdout:142: writing smithi138138488-29 from 1583497 to 2083190 tid 2 2023-06-27T12:38:21.494 INFO:tasks.rados.rados.0.smithi138.stdout:142: writing smithi138138488-29 from 2555481 to 2980668 tid 3 2023-06-27T12:38:21.494 INFO:tasks.rados.rados.0.smithi138.stdout:143: write (excl) oid 3 current snap is 7 2023-06-27T12:38:21.495 INFO:tasks.rados.rados.0.smithi138.stdout:143: seq_num 91 ranges {729437=527542,1925626=643351,3266551=262886} 2023-06-27T12:38:21.501 INFO:tasks.rados.rados.0.smithi138.stdout:143: writing smithi138138488-3 from 729437 to 1256979 tid 1 2023-06-27T12:38:21.510 INFO:tasks.rados.rados.0.smithi138.stdout:143: writing smithi138138488-3 from 1925626 to 2568977 tid 2 2023-06-27T12:38:21.516 INFO:tasks.rados.rados.0.smithi138.stdout:143: writing smithi138138488-3 from 3266551 to 3529437 tid 3 2023-06-27T12:38:21.517 INFO:tasks.rados.rados.0.smithi138.stdout:144: write oid 14 current snap is 7 2023-06-27T12:38:21.517 INFO:tasks.rados.rados.0.smithi138.stdout:144: seq_num 92 ranges {746244=719144,2100461=503362,3126680=419564} 2023-06-27T12:38:21.524 INFO:tasks.rados.rados.0.smithi138.stdout:144: writing smithi138138488-14 from 746244 to 1465388 tid 1 2023-06-27T12:38:21.531 INFO:tasks.rados.rados.0.smithi138.stdout:144: writing smithi138138488-14 from 2100461 to 2603823 tid 2 2023-06-27T12:38:21.536 INFO:tasks.rados.rados.0.smithi138.stdout:144: writing smithi138138488-14 from 3126680 to 3546244 tid 3 2023-06-27T12:38:21.537 INFO:tasks.rados.rados.0.smithi138.stdout:145: write oid 13 current snap is 7 2023-06-27T12:38:21.537 INFO:tasks.rados.rados.0.smithi138.stdout:145: seq_num 93 ranges {763051=794393,2158943=647020,3437750=125301} 2023-06-27T12:38:21.545 INFO:tasks.rados.rados.0.smithi138.stdout:145: writing smithi138138488-13 from 763051 to 1557444 tid 1 2023-06-27T12:38:21.552 INFO:tasks.rados.rados.0.smithi138.stdout:145: writing smithi138138488-13 from 2158943 to 2805963 tid 2 2023-06-27T12:38:21.557 INFO:tasks.rados.rados.0.smithi138.stdout:145: writing smithi138138488-13 from 3437750 to 3563051 tid 3 2023-06-27T12:38:21.557 INFO:tasks.rados.rados.0.smithi138.stdout:146: write oid 34 current snap is 7 2023-06-27T12:38:21.557 INFO:tasks.rados.rados.0.smithi138.stdout:146: seq_num 94 ranges {779858=469642,1817425=507031,2781526=586628} 2023-06-27T12:38:21.564 INFO:tasks.rados.rados.0.smithi138.stdout:146: writing smithi138138488-34 from 779858 to 1249500 tid 1 2023-06-27T12:38:21.570 INFO:tasks.rados.rados.0.smithi138.stdout:146: writing smithi138138488-34 from 1817425 to 2324456 tid 2 2023-06-27T12:38:21.577 INFO:tasks.rados.rados.0.smithi138.stdout:146: writing smithi138138488-34 from 2781526 to 3368154 tid 3 2023-06-27T12:38:21.577 INFO:tasks.rados.rados.0.smithi138.stdout:147: read oid 35 snap -1 2023-06-27T12:38:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:147: expect (ObjNum 84 snap 6 seq_num 84) 2023-06-27T12:38:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:148: read oid 47 snap -1 2023-06-27T12:38:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:148: expect (ObjNum 46 snap 0 seq_num 46) 2023-06-27T12:38:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:149: delete oid 49 current snap is 7 2023-06-27T12:38:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:140: expect (ObjNum 68 snap 3 seq_num 68) 2023-06-27T12:38:21.744 INFO:tasks.rados.rados.0.smithi138.stdout:137: expect (ObjNum 85 snap 6 seq_num 85) 2023-06-27T12:38:21.926 INFO:tasks.rados.rados.0.smithi138.stdout:139: expect (ObjNum 31 snap 0 seq_num 31) 2023-06-27T12:38:22.021 INFO:tasks.rados.rados.0.smithi138.stdout:138: expect (ObjNum 15 snap 0 seq_num 15) 2023-06-27T12:38:22.107 INFO:tasks.rados.rados.0.smithi138.stdout:141: finishing write tid 1 to smithi138138488-17 2023-06-27T12:38:22.107 INFO:tasks.rados.rados.0.smithi138.stdout:144: finishing write tid 1 to smithi138138488-14 2023-06-27T12:38:22.107 INFO:tasks.rados.rados.0.smithi138.stdout:141: finishing write tid 2 to smithi138138488-17 2023-06-27T12:38:22.107 INFO:tasks.rados.rados.0.smithi138.stdout:142: finishing write tid 1 to smithi138138488-29 2023-06-27T12:38:22.108 INFO:tasks.rados.rados.0.smithi138.stdout:137: done (12 left) 2023-06-27T12:38:22.108 INFO:tasks.rados.rados.0.smithi138.stdout:138: done (11 left) 2023-06-27T12:38:22.108 INFO:tasks.rados.rados.0.smithi138.stdout:139: done (10 left) 2023-06-27T12:38:22.108 INFO:tasks.rados.rados.0.smithi138.stdout:140: done (9 left) 2023-06-27T12:38:22.108 INFO:tasks.rados.rados.0.smithi138.stdout:149: done (8 left) 2023-06-27T12:38:22.108 INFO:tasks.rados.rados.0.smithi138.stdout:150: write (excl) oid 1 current snap is 7 2023-06-27T12:38:22.108 INFO:tasks.rados.rados.0.smithi138.stdout:141: finishing write tid 3 to smithi138138488-17 2023-06-27T12:38:22.109 INFO:tasks.rados.rados.0.smithi138.stdout:150: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2023-06-27T12:38:22.112 INFO:tasks.rados.rados.0.smithi138.stdout:150: writing smithi138138488-1 from 413472 to 1033612 tid 1 2023-06-27T12:38:22.119 INFO:tasks.rados.rados.0.smithi138.stdout:150: writing smithi138138488-1 from 1818036 to 2328736 tid 2 2023-06-27T12:38:22.127 INFO:tasks.rados.rados.0.smithi138.stdout:150: writing smithi138138488-1 from 3120019 to 3613472 tid 3 2023-06-27T12:38:22.127 INFO:tasks.rados.rados.0.smithi138.stdout:151: delete oid 36 current snap is 7 2023-06-27T12:38:22.127 INFO:tasks.rados.rados.0.smithi138.stdout:141: finishing write tid 4 to smithi138138488-17 2023-06-27T12:38:22.127 INFO:tasks.rados.rados.0.smithi138.stdout:141: finishing write tid 5 to smithi138138488-17 2023-06-27T12:38:22.128 INFO:tasks.rados.rados.0.smithi138.stdout:141: oid 17 updating version 0 to 25 2023-06-27T12:38:22.128 INFO:tasks.rados.rados.0.smithi138.stdout:141: oid 17 version 25 is already newer than 25 2023-06-27T12:38:22.128 INFO:tasks.rados.rados.0.smithi138.stdout:141: oid 17 version 25 is already newer than 25 2023-06-27T12:38:22.128 INFO:tasks.rados.rados.0.smithi138.stdout:141: oid 17 version 25 is already newer than 25 2023-06-27T12:38:22.128 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 25 (ObjNum 89 snap 7 seq_num 89) dirty exists 2023-06-27T12:38:22.128 INFO:tasks.rados.rados.0.smithi138.stdout:141: left oid 17 (ObjNum 89 snap 7 seq_num 89) 2023-06-27T12:38:22.128 INFO:tasks.rados.rados.0.smithi138.stdout:146: finishing write tid 1 to smithi138138488-34 2023-06-27T12:38:22.128 INFO:tasks.rados.rados.0.smithi138.stdout:145: finishing write tid 1 to smithi138138488-13 2023-06-27T12:38:22.129 INFO:tasks.rados.rados.0.smithi138.stdout:144: finishing write tid 2 to smithi138138488-14 2023-06-27T12:38:22.129 INFO:tasks.rados.rados.0.smithi138.stdout:148: expect (ObjNum 46 snap 0 seq_num 46) 2023-06-27T12:38:22.236 INFO:tasks.rados.rados.0.smithi138.stdout:142: finishing write tid 2 to smithi138138488-29 2023-06-27T12:38:22.236 INFO:tasks.rados.rados.0.smithi138.stdout:143: finishing write tid 1 to smithi138138488-3 2023-06-27T12:38:22.236 INFO:tasks.rados.rados.0.smithi138.stdout:142: finishing write tid 3 to smithi138138488-29 2023-06-27T12:38:22.236 INFO:tasks.rados.rados.0.smithi138.stdout:143: finishing write tid 2 to smithi138138488-3 2023-06-27T12:38:22.237 INFO:tasks.rados.rados.0.smithi138.stdout:146: finishing write tid 2 to smithi138138488-34 2023-06-27T12:38:22.237 INFO:tasks.rados.rados.0.smithi138.stdout:146: finishing write tid 3 to smithi138138488-34 2023-06-27T12:38:22.237 INFO:tasks.rados.rados.0.smithi138.stdout:146: finishing write tid 4 to smithi138138488-34 2023-06-27T12:38:22.237 INFO:tasks.rados.rados.0.smithi138.stdout:143: finishing write tid 3 to smithi138138488-3 2023-06-27T12:38:22.237 INFO:tasks.rados.rados.0.smithi138.stdout:146: finishing write tid 5 to smithi138138488-34 2023-06-27T12:38:22.237 INFO:tasks.rados.rados.0.smithi138.stdout:146: oid 34 updating version 0 to 40 2023-06-27T12:38:22.237 INFO:tasks.rados.rados.0.smithi138.stdout:146: oid 34 version 40 is already newer than 40 2023-06-27T12:38:22.238 INFO:tasks.rados.rados.0.smithi138.stdout:146: oid 34 version 40 is already newer than 40 2023-06-27T12:38:22.238 INFO:tasks.rados.rados.0.smithi138.stdout:146: oid 34 version 40 is already newer than 40 2023-06-27T12:38:22.238 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 40 (ObjNum 94 snap 7 seq_num 94) dirty exists 2023-06-27T12:38:22.238 INFO:tasks.rados.rados.0.smithi138.stdout:146: left oid 34 (ObjNum 94 snap 7 seq_num 94) 2023-06-27T12:38:22.238 INFO:tasks.rados.rados.0.smithi138.stdout:144: finishing write tid 3 to smithi138138488-14 2023-06-27T12:38:22.238 INFO:tasks.rados.rados.0.smithi138.stdout:144: finishing write tid 4 to smithi138138488-14 2023-06-27T12:38:22.238 INFO:tasks.rados.rados.0.smithi138.stdout:144: finishing write tid 5 to smithi138138488-14 2023-06-27T12:38:22.239 INFO:tasks.rados.rados.0.smithi138.stdout:144: oid 14 updating version 0 to 32 2023-06-27T12:38:22.239 INFO:tasks.rados.rados.0.smithi138.stdout:144: oid 14 version 32 is already newer than 31 2023-06-27T12:38:22.239 INFO:tasks.rados.rados.0.smithi138.stdout:144: oid 14 version 32 is already newer than 32 2023-06-27T12:38:22.239 INFO:tasks.rados.rados.0.smithi138.stdout:144: oid 14 version 32 is already newer than 32 2023-06-27T12:38:22.239 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 32 (ObjNum 92 snap 7 seq_num 92) dirty exists 2023-06-27T12:38:22.239 INFO:tasks.rados.rados.0.smithi138.stdout:144: left oid 14 (ObjNum 92 snap 7 seq_num 92) 2023-06-27T12:38:22.239 INFO:tasks.rados.rados.0.smithi138.stdout:142: finishing write tid 4 to smithi138138488-29 2023-06-27T12:38:22.239 INFO:tasks.rados.rados.0.smithi138.stdout:145: finishing write tid 2 to smithi138138488-13 2023-06-27T12:38:22.240 INFO:tasks.rados.rados.0.smithi138.stdout:142: finishing write tid 5 to smithi138138488-29 2023-06-27T12:38:22.240 INFO:tasks.rados.rados.0.smithi138.stdout:142: oid 29 updating version 0 to 10 2023-06-27T12:38:22.240 INFO:tasks.rados.rados.0.smithi138.stdout:142: oid 29 version 10 is already newer than 10 2023-06-27T12:38:22.240 INFO:tasks.rados.rados.0.smithi138.stdout:142: oid 29 version 10 is already newer than 10 2023-06-27T12:38:22.240 INFO:tasks.rados.rados.0.smithi138.stdout:142: oid 29 version 10 is already newer than 7 2023-06-27T12:38:22.240 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 10 (ObjNum 90 snap 7 seq_num 90) dirty exists 2023-06-27T12:38:22.241 INFO:tasks.rados.rados.0.smithi138.stdout:142: left oid 29 (ObjNum 90 snap 7 seq_num 90) 2023-06-27T12:38:22.241 INFO:tasks.rados.rados.0.smithi138.stdout:145: finishing write tid 3 to smithi138138488-13 2023-06-27T12:38:22.241 INFO:tasks.rados.rados.0.smithi138.stdout:145: finishing write tid 4 to smithi138138488-13 2023-06-27T12:38:22.242 INFO:tasks.rados.rados.0.smithi138.stdout:145: finishing write tid 5 to smithi138138488-13 2023-06-27T12:38:22.242 INFO:tasks.rados.rados.0.smithi138.stdout:145: oid 13 updating version 0 to 47 2023-06-27T12:38:22.242 INFO:tasks.rados.rados.0.smithi138.stdout:145: oid 13 version 47 is already newer than 47 2023-06-27T12:38:22.242 INFO:tasks.rados.rados.0.smithi138.stdout:145: oid 13 version 47 is already newer than 47 2023-06-27T12:38:22.242 INFO:tasks.rados.rados.0.smithi138.stdout:145: oid 13 version 47 is already newer than 47 2023-06-27T12:38:22.242 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 47 (ObjNum 93 snap 7 seq_num 93) dirty exists 2023-06-27T12:38:22.243 INFO:tasks.rados.rados.0.smithi138.stdout:145: left oid 13 (ObjNum 93 snap 7 seq_num 93) 2023-06-27T12:38:22.243 INFO:tasks.rados.rados.0.smithi138.stdout:141: done (9 left) 2023-06-27T12:38:22.243 INFO:tasks.rados.rados.0.smithi138.stdout:142: done (8 left) 2023-06-27T12:38:22.243 INFO:tasks.rados.rados.0.smithi138.stdout:144: done (7 left) 2023-06-27T12:38:22.243 INFO:tasks.rados.rados.0.smithi138.stdout:145: done (6 left) 2023-06-27T12:38:22.243 INFO:tasks.rados.rados.0.smithi138.stdout:146: done (5 left) 2023-06-27T12:38:22.243 INFO:tasks.rados.rados.0.smithi138.stdout:148: done (4 left) 2023-06-27T12:38:22.243 INFO:tasks.rados.rados.0.smithi138.stdout:151: done (3 left) 2023-06-27T12:38:22.244 INFO:tasks.rados.rados.0.smithi138.stdout:152: write (excl) oid 19 current snap is 7 2023-06-27T12:38:22.244 INFO:tasks.rados.rados.0.smithi138.stdout:143: finishing write tid 4 to smithi138138488-3 2023-06-27T12:38:22.244 INFO:tasks.rados.rados.0.smithi138.stdout:152: seq_num 98 ranges {447086=770638,1935000=798016,3458512=188574} 2023-06-27T12:38:22.244 INFO:tasks.rados.rados.0.smithi138.stdout:152: writing smithi138138488-19 from 447086 to 1217724 tid 1 2023-06-27T12:38:22.250 INFO:tasks.rados.rados.0.smithi138.stdout:152: writing smithi138138488-19 from 1935000 to 2733016 tid 2 2023-06-27T12:38:22.256 INFO:tasks.rados.rados.0.smithi138.stdout:152: writing smithi138138488-19 from 3458512 to 3647086 tid 3 2023-06-27T12:38:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:153: read oid 36 snap -1 2023-06-27T12:38:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:153: expect deleted 2023-06-27T12:38:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:154: snap_create 2023-06-27T12:38:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:147: expect (ObjNum 84 snap 6 seq_num 84) 2023-06-27T12:38:22.471 INFO:tasks.rados.rados.0.smithi138.stdout:143: finishing write tid 5 to smithi138138488-3 2023-06-27T12:38:22.471 INFO:tasks.rados.rados.0.smithi138.stdout:143: oid 3 updating version 0 to 24 2023-06-27T12:38:22.472 INFO:tasks.rados.rados.0.smithi138.stdout:143: oid 3 version 24 is already newer than 24 2023-06-27T12:38:22.472 INFO:tasks.rados.rados.0.smithi138.stdout:143: oid 3 version 24 is already newer than 21 2023-06-27T12:38:22.472 INFO:tasks.rados.rados.0.smithi138.stdout:143: oid 3 version 24 is already newer than 24 2023-06-27T12:38:22.472 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 24 (ObjNum 91 snap 7 seq_num 91) dirty exists 2023-06-27T12:38:22.472 INFO:tasks.rados.rados.0.smithi138.stdout:143: left oid 3 (ObjNum 91 snap 7 seq_num 91) 2023-06-27T12:38:22.472 INFO:tasks.rados.rados.0.smithi138.stdout:150: finishing write tid 1 to smithi138138488-1 2023-06-27T12:38:22.473 INFO:tasks.rados.rados.0.smithi138.stdout:150: finishing write tid 2 to smithi138138488-1 2023-06-27T12:38:22.473 INFO:tasks.rados.rados.0.smithi138.stdout:150: finishing write tid 3 to smithi138138488-1 2023-06-27T12:38:22.473 INFO:tasks.rados.rados.0.smithi138.stdout:150: finishing write tid 4 to smithi138138488-1 2023-06-27T12:38:22.473 INFO:tasks.rados.rados.0.smithi138.stdout:150: finishing write tid 5 to smithi138138488-1 2023-06-27T12:38:22.473 INFO:tasks.rados.rados.0.smithi138.stdout:150: oid 1 updating version 0 to 33 2023-06-27T12:38:22.473 INFO:tasks.rados.rados.0.smithi138.stdout:150: oid 1 updating version 33 to 36 2023-06-27T12:38:22.473 INFO:tasks.rados.rados.0.smithi138.stdout:150: oid 1 version 36 is already newer than 36 2023-06-27T12:38:22.473 INFO:tasks.rados.rados.0.smithi138.stdout:150: oid 1 version 36 is already newer than 36 2023-06-27T12:38:22.474 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 36 (ObjNum 96 snap 7 seq_num 96) dirty exists 2023-06-27T12:38:22.474 INFO:tasks.rados.rados.0.smithi138.stdout:150: left oid 1 (ObjNum 96 snap 7 seq_num 96) 2023-06-27T12:38:22.474 INFO:tasks.rados.rados.0.smithi138.stdout:152: finishing write tid 1 to smithi138138488-19 2023-06-27T12:38:22.474 INFO:tasks.rados.rados.0.smithi138.stdout:152: finishing write tid 2 to smithi138138488-19 2023-06-27T12:38:22.474 INFO:tasks.rados.rados.0.smithi138.stdout:152: finishing write tid 3 to smithi138138488-19 2023-06-27T12:38:22.474 INFO:tasks.rados.rados.0.smithi138.stdout:152: finishing write tid 4 to smithi138138488-19 2023-06-27T12:38:22.474 INFO:tasks.rados.rados.0.smithi138.stdout:152: finishing write tid 5 to smithi138138488-19 2023-06-27T12:38:22.474 INFO:tasks.rados.rados.0.smithi138.stdout:152: oid 19 updating version 0 to 37 2023-06-27T12:38:22.475 INFO:tasks.rados.rados.0.smithi138.stdout:152: oid 19 updating version 37 to 40 2023-06-27T12:38:22.475 INFO:tasks.rados.rados.0.smithi138.stdout:152: oid 19 version 40 is already newer than 40 2023-06-27T12:38:22.475 INFO:tasks.rados.rados.0.smithi138.stdout:152: oid 19 version 40 is already newer than 40 2023-06-27T12:38:22.475 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 40 (ObjNum 98 snap 7 seq_num 98) dirty exists 2023-06-27T12:38:22.475 INFO:tasks.rados.rados.0.smithi138.stdout:152: left oid 19 (ObjNum 98 snap 7 seq_num 98) 2023-06-27T12:38:22.475 INFO:tasks.rados.rados.0.smithi138.stdout:143: done (5 left) 2023-06-27T12:38:22.475 INFO:tasks.rados.rados.0.smithi138.stdout:147: done (4 left) 2023-06-27T12:38:22.475 INFO:tasks.rados.rados.0.smithi138.stdout:150: done (3 left) 2023-06-27T12:38:22.476 INFO:tasks.rados.rados.0.smithi138.stdout:152: done (2 left) 2023-06-27T12:38:22.476 INFO:tasks.rados.rados.0.smithi138.stdout:153: done (1 left) 2023-06-27T12:38:22.476 INFO:tasks.rados.rados.0.smithi138.stdout:154: done (0 left) 2023-06-27T12:38:22.476 INFO:tasks.rados.rados.0.smithi138.stdout:155: read oid 33 snap -1 2023-06-27T12:38:22.476 INFO:tasks.rados.rados.0.smithi138.stdout:155: expect (ObjNum 32 snap 0 seq_num 32) 2023-06-27T12:38:22.476 INFO:tasks.rados.rados.0.smithi138.stdout:156: write (excl) oid 26 current snap is 8 2023-06-27T12:38:22.476 INFO:tasks.rados.rados.0.smithi138.stdout:156: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2023-06-27T12:38:22.477 INFO:tasks.rados.rados.0.smithi138.stdout:156: writing smithi138138488-26 from 463893 to 1026133 tid 1 2023-06-27T12:38:22.483 INFO:tasks.rados.rados.0.smithi138.stdout:156: writing smithi138138488-26 from 1593482 to 2251509 tid 2 2023-06-27T12:38:22.489 INFO:tasks.rados.rados.0.smithi138.stdout:156: writing smithi138138488-26 from 2685935 to 3245276 tid 3 2023-06-27T12:38:22.489 INFO:tasks.rados.rados.0.smithi138.stdout:157: delete oid 5 current snap is 8 2023-06-27T12:38:22.509 INFO:tasks.rados.rados.0.smithi138.stdout:156: finishing write tid 1 to smithi138138488-26 2023-06-27T12:38:22.509 INFO:tasks.rados.rados.0.smithi138.stdout:155: expect (ObjNum 32 snap 0 seq_num 32) 2023-06-27T12:38:22.606 INFO:tasks.rados.rados.0.smithi138.stdout:156: finishing write tid 2 to smithi138138488-26 2023-06-27T12:38:22.607 INFO:tasks.rados.rados.0.smithi138.stdout:156: finishing write tid 3 to smithi138138488-26 2023-06-27T12:38:22.607 INFO:tasks.rados.rados.0.smithi138.stdout:156: finishing write tid 4 to smithi138138488-26 2023-06-27T12:38:22.607 INFO:tasks.rados.rados.0.smithi138.stdout:156: finishing write tid 5 to smithi138138488-26 2023-06-27T12:38:22.607 INFO:tasks.rados.rados.0.smithi138.stdout:156: oid 26 updating version 0 to 25 2023-06-27T12:38:22.607 INFO:tasks.rados.rados.0.smithi138.stdout:156: oid 26 updating version 25 to 29 2023-06-27T12:38:22.607 INFO:tasks.rados.rados.0.smithi138.stdout:156: oid 26 version 29 is already newer than 29 2023-06-27T12:38:22.608 INFO:tasks.rados.rados.0.smithi138.stdout:156: oid 26 version 29 is already newer than 29 2023-06-27T12:38:22.608 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 29 (ObjNum 99 snap 8 seq_num 99) dirty exists 2023-06-27T12:38:22.608 INFO:tasks.rados.rados.0.smithi138.stdout:156: left oid 26 (ObjNum 99 snap 8 seq_num 99) 2023-06-27T12:38:22.608 INFO:tasks.rados.rados.0.smithi138.stdout:155: done (2 left) 2023-06-27T12:38:22.608 INFO:tasks.rados.rados.0.smithi138.stdout:156: done (1 left) 2023-06-27T12:38:22.608 INFO:tasks.rados.rados.0.smithi138.stdout:157: done (0 left) 2023-06-27T12:38:22.608 INFO:tasks.rados.rados.0.smithi138.stdout:158: read oid 45 snap 7 2023-06-27T12:38:22.608 INFO:tasks.rados.rados.0.smithi138.stdout:158: expect (ObjNum 70 snap 3 seq_num 70) 2023-06-27T12:38:22.609 INFO:tasks.rados.rados.0.smithi138.stdout:159: read oid 31 snap -1 2023-06-27T12:38:22.609 INFO:tasks.rados.rados.0.smithi138.stdout:159: expect (ObjNum 30 snap 0 seq_num 30) 2023-06-27T12:38:22.609 INFO:tasks.rados.rados.0.smithi138.stdout:160: snap_create 2023-06-27T12:38:22.635 INFO:tasks.rados.rados.0.smithi138.stdout:158: expect (ObjNum 70 snap 3 seq_num 70) 2023-06-27T12:38:22.849 INFO:tasks.rados.rados.0.smithi138.stdout:159: expect (ObjNum 30 snap 0 seq_num 30) 2023-06-27T12:38:22.850 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:22.975 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:22.970+0000 7f3ff49ef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:22.978 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:22.974+0000 7f3ff49ef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:23.461 INFO:tasks.rados.rados.0.smithi138.stdout:158: done (2 left) 2023-06-27T12:38:23.462 INFO:tasks.rados.rados.0.smithi138.stdout:159: done (1 left) 2023-06-27T12:38:23.462 INFO:tasks.rados.rados.0.smithi138.stdout:160: done (0 left) 2023-06-27T12:38:23.462 INFO:tasks.rados.rados.0.smithi138.stdout:161: snap_remove snap 8 2023-06-27T12:38:24.177 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:24.177 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:24.178 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:24.320 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:24.313+0000 7fba1afe1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:24.323 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:24.319+0000 7fba1afe1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:24.467 INFO:tasks.rados.rados.0.smithi138.stdout:161: done (0 left) 2023-06-27T12:38:24.467 INFO:tasks.rados.rados.0.smithi138.stdout:162: write (excl) oid 26 current snap is 9 2023-06-27T12:38:24.467 INFO:tasks.rados.rados.0.smithi138.stdout:162: seq_num 101 ranges {497507=712738,1710446=661696,3140781=556726} 2023-06-27T12:38:24.477 INFO:tasks.rados.rados.0.smithi138.stdout:162: writing smithi138138488-26 from 497507 to 1210245 tid 1 2023-06-27T12:38:24.487 INFO:tasks.rados.rados.0.smithi138.stdout:162: writing smithi138138488-26 from 1710446 to 2372142 tid 2 2023-06-27T12:38:24.497 INFO:tasks.rados.rados.0.smithi138.stdout:162: writing smithi138138488-26 from 3140781 to 3697507 tid 3 2023-06-27T12:38:24.498 INFO:tasks.rados.rados.0.smithi138.stdout:163: snap_remove snap 7 2023-06-27T12:38:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:24.534 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:24.546 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-06-27T12:38:24.462785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":132675096,"num_objects":70,"num_object_clones":19,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":406,"num_read_kb":171902,"num_write":519,"num_write_kb":110959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":455,"ondisk_log_size":455,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":354724,"kb_used_data":235812,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377132636,"statfs":{"total":386547056640,"available":386183819264,"internally_reserved":0,"allocated":241471488,"data_stored":248832023,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11899356,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":67,"num_read_kb":50391,"num_write":75,"num_write_kb":23310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013835"},"pg_stats":[{"pgid":"3.f","version":"18'4","reported_seq":51,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.483873+0000","last_change":"2023-06-27T12:38:21.489585+0000","last_active":"2023-06-27T12:38:22.483873+0000","last_peered":"2023-06-27T12:38:22.483873+0000","last_clean":"2023-06-27T12:38:22.483873+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:22.483873+0000","last_undegraded":"2023-06-27T12:38:22.483873+0000","last_fullsized":"2023-06-27T12:38:22.483873+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"33'25","reported_seq":84,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-27T12:38:23.467824+0000","last_change":"2023-06-27T12:38:21.497785+0000","last_active":"2023-06-27T12:38:23.467824+0000","last_peered":"2023-06-27T12:38:23.467824+0000","last_clean":"2023-06-27T12:38:23.467824+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:23.467824+0000","last_undegraded":"2023-06-27T12:38:23.467824+0000","last_fullsized":"2023-06-27T12:38:23.467824+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11781575,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":17648,"num_write":25,"num_write_kb":7250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'28","reported_seq":88,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.620464+0000","last_change":"2023-06-27T12:38:21.492301+0000","last_active":"2023-06-27T12:38:22.620464+0000","last_peered":"2023-06-27T12:38:22.620464+0000","last_clean":"2023-06-27T12:38:22.620464+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:22.620464+0000","last_undegraded":"2023-06-27T12:38:22.620464+0000","last_fullsized":"2023-06-27T12:38:22.620464+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10184910,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":12452,"num_write":35,"num_write_kb":9601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"32'22","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.501197+0000","last_change":"2023-06-27T12:38:21.498000+0000","last_active":"2023-06-27T12:38:22.501197+0000","last_peered":"2023-06-27T12:38:22.501197+0000","last_clean":"2023-06-27T12:38:22.501197+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:22.501197+0000","last_undegraded":"2023-06-27T12:38:22.501197+0000","last_fullsized":"2023-06-27T12:38:22.501197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9025260,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":18,"num_read_kb":3227,"num_write":22,"num_write_kb":6297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"18'6","reported_seq":58,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.519904+0000","last_change":"2023-06-27T12:38:21.504852+0000","last_active":"2023-06-27T12:38:22.519904+0000","last_peered":"2023-06-27T12:38:22.519904+0000","last_clean":"2023-06-27T12:38:22.519904+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:22.519904+0000","last_undegraded":"2023-06-27T12:38:22.519904+0000","last_fullsized":"2023-06-27T12:38:22.519904+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4352947,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":11423,"num_write":8,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"33'40","reported_seq":97,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.625346+0000","last_change":"2023-06-27T12:38:21.488624+0000","last_active":"2023-06-27T12:38:22.625346+0000","last_peered":"2023-06-27T12:38:22.625346+0000","last_clean":"2023-06-27T12:38:22.625346+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:22.625346+0000","last_undegraded":"2023-06-27T12:38:22.625346+0000","last_fullsized":"2023-06-27T12:38:22.625346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14941258,"num_objects":8,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":16658,"num_write":40,"num_write_kb":10991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"31'31","reported_seq":83,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.481659+0000","last_change":"2023-06-27T12:38:21.496995+0000","last_active":"2023-06-27T12:38:22.481659+0000","last_peered":"2023-06-27T12:38:22.481659+0000","last_clean":"2023-06-27T12:38:22.481659+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:22.481659+0000","last_undegraded":"2023-06-27T12:38:22.481659+0000","last_fullsized":"2023-06-27T12:38:22.481659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8739541,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":8974,"num_write":30,"num_write_kb":8934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"28'8","reported_seq":63,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.619583+0000","last_change":"2023-06-27T12:38:21.486899+0000","last_active":"2023-06-27T12:38:22.619583+0000","last_peered":"2023-06-27T12:38:22.619583+0000","last_clean":"2023-06-27T12:38:22.619583+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:22.619583+0000","last_undegraded":"2023-06-27T12:38:22.619583+0000","last_fullsized":"2023-06-27T12:38:22.619583+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3327753,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":11710,"num_write":9,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-27T12:38:23.476221+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:23.476221+0000","last_peered":"2023-06-27T12:38:23.476221+0000","last_clean":"2023-06-27T12:38:23.476221+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:23.476221+0000","last_undegraded":"2023-06-27T12:38:23.476221+0000","last_fullsized":"2023-06-27T12:38:23.476221+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":46,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.624525+0000","last_change":"2023-06-27T12:38:21.492193+0000","last_active":"2023-06-27T12:38:22.624525+0000","last_peered":"2023-06-27T12:38:22.624525+0000","last_clean":"2023-06-27T12:38:22.624525+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:22.624525+0000","last_undegraded":"2023-06-27T12:38:22.624525+0000","last_fullsized":"2023-06-27T12:38:22.624525+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.482506+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:22.482506+0000","last_peered":"2023-06-27T12:38:22.482506+0000","last_clean":"2023-06-27T12:38:22.482506+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:22.482506+0000","last_undegraded":"2023-06-27T12:38:22.482506+0000","last_fullsized":"2023-06-27T12:38:22.482506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'19","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-27T12:38:23.474144+0000","last_change":"2023-06-27T12:38:21.495158+0000","last_active":"2023-06-27T12:38:23.474144+0000","last_peered":"2023-06-27T12:38:23.474144+0000","last_clean":"2023-06-27T12:38:23.474144+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:23.474144+0000","last_undegraded":"2023-06-27T12:38:23.474144+0000","last_fullsized":"2023-06-27T12:38:23.474144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6151296,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2665,"num_write":21,"num_write_kb":5953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.621744+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:22.621744+0000","last_peered":"2023-06-27T12:38:22.621744+0000","last_clean":"2023-06-27T12:38:22.621744+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:22.621744+0000","last_undegraded":"2023-06-27T12:38:22.621744+0000","last_fullsized":"2023-06-27T12:38:22.621744+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"33'47","reported_seq":114,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.609251+0000","last_change":"2023-06-27T12:38:21.482554+0000","last_active":"2023-06-27T12:38:22.609251+0000","last_peered":"2023-06-27T12:38:22.609251+0000","last_clean":"2023-06-27T12:38:22.609251+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:22.609251+0000","last_undegraded":"2023-06-27T12:38:22.609251+0000","last_fullsized":"2023-06-27T12:38:22.609251+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16100941,"num_objects":7,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":31223,"num_write":53,"num_write_kb":15611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.623626+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:22.623626+0000","last_peered":"2023-06-27T12:38:22.623626+0000","last_clean":"2023-06-27T12:38:22.623626+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:22.623626+0000","last_undegraded":"2023-06-27T12:38:22.623626+0000","last_fullsized":"2023-06-27T12:38:22.623626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":172,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.521300+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:22.521300+0000","last_peered":"2023-06-27T12:38:22.521300+0000","last_clean":"2023-06-27T12:38:22.521300+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:22.521300+0000","last_undegraded":"2023-06-27T12:38:22.521300+0000","last_fullsized":"2023-06-27T12:38:22.521300+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"33'40","reported_seq":97,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.619959+0000","last_change":"2023-06-27T12:38:21.493518+0000","last_active":"2023-06-27T12:38:22.619959+0000","last_peered":"2023-06-27T12:38:22.619959+0000","last_clean":"2023-06-27T12:38:22.619959+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:22.619959+0000","last_undegraded":"2023-06-27T12:38:22.619959+0000","last_fullsized":"2023-06-27T12:38:22.619959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15277398,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":33,"num_read_kb":8908,"num_write":44,"num_write_kb":12120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.480947+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:22.480947+0000","last_peered":"2023-06-27T12:38:22.480947+0000","last_clean":"2023-06-27T12:38:22.480947+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:22.480947+0000","last_undegraded":"2023-06-27T12:38:22.480947+0000","last_fullsized":"2023-06-27T12:38:22.480947+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"22'16","reported_seq":74,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.505502+0000","last_change":"2023-06-27T12:38:21.483804+0000","last_active":"2023-06-27T12:38:22.505502+0000","last_peered":"2023-06-27T12:38:22.505502+0000","last_clean":"2023-06-27T12:38:22.505502+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:22.505502+0000","last_undegraded":"2023-06-27T12:38:22.505502+0000","last_fullsized":"2023-06-27T12:38:22.505502+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7781542,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":17,"num_read_kb":18623,"num_write":20,"num_write_kb":5449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.515072+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:22.515072+0000","last_peered":"2023-06-27T12:38:22.515072+0000","last_clean":"2023-06-27T12:38:22.515072+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:22.515072+0000","last_undegraded":"2023-06-27T12:38:22.515072+0000","last_fullsized":"2023-06-27T12:38:22.515072+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"34'29","reported_seq":86,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-27T12:38:23.471101+0000","last_change":"2023-06-27T12:38:21.485255+0000","last_active":"2023-06-27T12:38:23.471101+0000","last_peered":"2023-06-27T12:38:23.471101+0000","last_clean":"2023-06-27T12:38:23.471101+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:23.471101+0000","last_undegraded":"2023-06-27T12:38:23.471101+0000","last_fullsized":"2023-06-27T12:38:23.471101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12571504,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":9847,"num_write":31,"num_write_kb":9200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"33'10","reported_seq":59,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.502169+0000","last_change":"2023-06-27T12:38:21.482803+0000","last_active":"2023-06-27T12:38:22.502169+0000","last_peered":"2023-06-27T12:38:22.502169+0000","last_clean":"2023-06-27T12:38:22.502169+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:22.502169+0000","last_undegraded":"2023-06-27T12:38:22.502169+0000","last_fullsized":"2023-06-27T12:38:22.502169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5915998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":3466,"num_write":13,"num_write_kb":3651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-27T12:38:23.475233+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:23.475233+0000","last_peered":"2023-06-27T12:38:23.475233+0000","last_clean":"2023-06-27T12:38:23.475233+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:23.475233+0000","last_undegraded":"2023-06-27T12:38:23.475233+0000","last_fullsized":"2023-06-27T12:38:23.475233+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-27T12:38:22.625806+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:22.625806+0000","last_peered":"2023-06-27T12:38:22.625806+0000","last_clean":"2023-06-27T12:38:22.625806+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:22.625806+0000","last_undegraded":"2023-06-27T12:38:22.625806+0000","last_fullsized":"2023-06-27T12:38:22.625806+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"33'33","reported_seq":87,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-27T12:38:23.473562+0000","last_change":"2023-06-27T12:38:21.485803+0000","last_active":"2023-06-27T12:38:23.473562+0000","last_peered":"2023-06-27T12:38:23.473562+0000","last_clean":"2023-06-27T12:38:23.473562+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:23.473562+0000","last_undegraded":"2023-06-27T12:38:23.473562+0000","last_fullsized":"2023-06-27T12:38:23.473562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5932805,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":20,"num_read_kb":14986,"num_write":27,"num_write_kb":8065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":132084728,"num_objects":68,"num_object_clones":19,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":303,"num_read_kb":171811,"num_write":382,"num_write_kb":109065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":358,"ondisk_log_size":358,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92996,"kb_used_data":62432,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94278844,"statfs":{"total":96636764160,"available":96541536256,"internally_reserved":0,"allocated":63930368,"data_stored":65090419,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83312,"kb_used_data":56488,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94288528,"statfs":{"total":96636764160,"available":96551452672,"internally_reserved":0,"allocated":57843712,"data_stored":59792142,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705695,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":85516,"kb_used_data":54716,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94286324,"statfs":{"total":96636764160,"available":96549195776,"internally_reserved":0,"allocated":56029184,"data_stored":58409001,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92900,"kb_used_data":62176,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94278940,"statfs":{"total":96636764160,"available":96541634560,"internally_reserved":0,"allocated":63668224,"data_stored":65540461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:25.047 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:25.176 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:25.172+0000 7f3782870700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:25.179 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:25.176+0000 7f3782870700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:25.469 INFO:tasks.rados.rados.0.smithi138.stdout:163: done (1 left) 2023-06-27T12:38:25.469 INFO:tasks.rados.rados.0.smithi138.stdout:164: read oid 7 snap -1 2023-06-27T12:38:25.470 INFO:tasks.rados.rados.0.smithi138.stdout:164: expect (ObjNum 67 snap 3 seq_num 67) 2023-06-27T12:38:25.470 INFO:tasks.rados.rados.0.smithi138.stdout:165: delete oid 12 current snap is 9 2023-06-27T12:38:25.470 INFO:tasks.rados.rados.0.smithi138.stdout:162: finishing write tid 1 to smithi138138488-26 2023-06-27T12:38:25.470 INFO:tasks.rados.rados.0.smithi138.stdout:162: finishing write tid 2 to smithi138138488-26 2023-06-27T12:38:25.471 INFO:tasks.rados.rados.0.smithi138.stdout:162: finishing write tid 3 to smithi138138488-26 2023-06-27T12:38:25.471 INFO:tasks.rados.rados.0.smithi138.stdout:162: finishing write tid 4 to smithi138138488-26 2023-06-27T12:38:25.471 INFO:tasks.rados.rados.0.smithi138.stdout:162: finishing write tid 5 to smithi138138488-26 2023-06-27T12:38:25.471 INFO:tasks.rados.rados.0.smithi138.stdout:162: oid 26 updating version 0 to 33 2023-06-27T12:38:25.471 INFO:tasks.rados.rados.0.smithi138.stdout:162: oid 26 version 33 is already newer than 33 2023-06-27T12:38:25.472 INFO:tasks.rados.rados.0.smithi138.stdout:162: oid 26 version 33 is already newer than 30 2023-06-27T12:38:25.472 INFO:tasks.rados.rados.0.smithi138.stdout:162: oid 26 version 33 is already newer than 33 2023-06-27T12:38:25.472 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 33 (ObjNum 101 snap 9 seq_num 101) dirty exists 2023-06-27T12:38:25.472 INFO:tasks.rados.rados.0.smithi138.stdout:162: left oid 26 (ObjNum 101 snap 9 seq_num 101) 2023-06-27T12:38:25.559 INFO:tasks.rados.rados.0.smithi138.stdout:164: expect (ObjNum 67 snap 3 seq_num 67) 2023-06-27T12:38:25.761 INFO:tasks.rados.rados.0.smithi138.stdout:162: done (2 left) 2023-06-27T12:38:25.761 INFO:tasks.rados.rados.0.smithi138.stdout:164: done (1 left) 2023-06-27T12:38:25.762 INFO:tasks.rados.rados.0.smithi138.stdout:165: done (0 left) 2023-06-27T12:38:25.762 INFO:tasks.rados.rados.0.smithi138.stdout:166: delete oid 48 current snap is 9 2023-06-27T12:38:25.807 INFO:tasks.rados.rados.0.smithi138.stdout:166: done (0 left) 2023-06-27T12:38:25.807 INFO:tasks.rados.rados.0.smithi138.stdout:167: delete oid 11 current snap is 9 2023-06-27T12:38:25.854 INFO:tasks.rados.rados.0.smithi138.stdout:167: done (0 left) 2023-06-27T12:38:25.855 INFO:tasks.rados.rados.0.smithi138.stdout:168: write (excl) oid 44 current snap is 9 2023-06-27T12:38:25.855 INFO:tasks.rados.rados.0.smithi138.stdout:168: seq_num 105 ranges {564735=613734,1828021=669034,3134120=630615} 2023-06-27T12:38:25.861 INFO:tasks.rados.rados.0.smithi138.stdout:168: writing smithi138138488-44 from 564735 to 1178469 tid 1 2023-06-27T12:38:25.868 INFO:tasks.rados.rados.0.smithi138.stdout:168: writing smithi138138488-44 from 1828021 to 2497055 tid 2 2023-06-27T12:38:25.875 INFO:tasks.rados.rados.0.smithi138.stdout:168: writing smithi138138488-44 from 3134120 to 3764735 tid 3 2023-06-27T12:38:25.875 INFO:tasks.rados.rados.0.smithi138.stdout:169: write oid 2 current snap is 9 2023-06-27T12:38:25.876 INFO:tasks.rados.rados.0.smithi138.stdout:169: seq_num 106 ranges {581542=688983,1886503=412692,2761543=670951} 2023-06-27T12:38:25.883 INFO:tasks.rados.rados.0.smithi138.stdout:169: writing smithi138138488-2 from 581542 to 1270525 tid 1 2023-06-27T12:38:25.888 INFO:tasks.rados.rados.0.smithi138.stdout:169: writing smithi138138488-2 from 1886503 to 2299195 tid 2 2023-06-27T12:38:25.895 INFO:tasks.rados.rados.0.smithi138.stdout:169: writing smithi138138488-2 from 2761543 to 3432494 tid 3 2023-06-27T12:38:25.896 INFO:tasks.rados.rados.0.smithi138.stdout:170: read oid 21 snap -1 2023-06-27T12:38:25.896 INFO:tasks.rados.rados.0.smithi138.stdout:170: expect (ObjNum 20 snap 0 seq_num 20) 2023-06-27T12:38:25.896 INFO:tasks.rados.rados.0.smithi138.stdout:171: write oid 17 current snap is 9 2023-06-27T12:38:25.897 INFO:tasks.rados.rados.0.smithi138.stdout:171: seq_num 107 ranges {598349=480585,1544985=672703,2905319=482223} 2023-06-27T12:38:25.901 INFO:tasks.rados.rados.0.smithi138.stdout:171: writing smithi138138488-17 from 598349 to 1078934 tid 1 2023-06-27T12:38:25.908 INFO:tasks.rados.rados.0.smithi138.stdout:171: writing smithi138138488-17 from 1544985 to 2217688 tid 2 2023-06-27T12:38:25.916 INFO:tasks.rados.rados.0.smithi138.stdout:171: writing smithi138138488-17 from 2905319 to 3387542 tid 3 2023-06-27T12:38:25.916 INFO:tasks.rados.rados.0.smithi138.stdout:172: write (excl) oid 15 current snap is 9 2023-06-27T12:38:25.917 INFO:tasks.rados.rados.0.smithi138.stdout:172: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2023-06-27T12:38:25.923 INFO:tasks.rados.rados.0.smithi138.stdout:172: writing smithi138138488-15 from 615156 to 1170990 tid 1 2023-06-27T12:38:25.929 INFO:tasks.rados.rados.0.smithi138.stdout:172: writing smithi138138488-15 from 1603467 to 2019828 tid 2 2023-06-27T12:38:25.938 INFO:tasks.rados.rados.0.smithi138.stdout:172: writing smithi138138488-15 from 2816389 to 3509884 tid 3 2023-06-27T12:38:25.938 INFO:tasks.rados.rados.0.smithi138.stdout:173: read oid 9 snap -1 2023-06-27T12:38:25.938 INFO:tasks.rados.rados.0.smithi138.stdout:173: expect (ObjNum 8 snap 0 seq_num 8) 2023-06-27T12:38:25.938 INFO:tasks.rados.rados.0.smithi138.stdout:174: write (excl) oid 39 current snap is 9 2023-06-27T12:38:25.939 INFO:tasks.rados.rados.0.smithi138.stdout:170: expect (ObjNum 20 snap 0 seq_num 20) 2023-06-27T12:38:26.036 INFO:tasks.rados.rados.0.smithi138.stdout:168: finishing write tid 1 to smithi138138488-44 2023-06-27T12:38:26.037 INFO:tasks.rados.rados.0.smithi138.stdout:169: finishing write tid 1 to smithi138138488-2 2023-06-27T12:38:26.037 INFO:tasks.rados.rados.0.smithi138.stdout:169: finishing write tid 2 to smithi138138488-2 2023-06-27T12:38:26.037 INFO:tasks.rados.rados.0.smithi138.stdout:173: expect (ObjNum 8 snap 0 seq_num 8) 2023-06-27T12:38:26.118 INFO:tasks.rados.rados.0.smithi138.stdout:169: finishing write tid 3 to smithi138138488-2 2023-06-27T12:38:26.119 INFO:tasks.rados.rados.0.smithi138.stdout:169: finishing write tid 4 to smithi138138488-2 2023-06-27T12:38:26.119 INFO:tasks.rados.rados.0.smithi138.stdout:169: finishing write tid 5 to smithi138138488-2 2023-06-27T12:38:26.119 INFO:tasks.rados.rados.0.smithi138.stdout:169: oid 2 updating version 0 to 51 2023-06-27T12:38:26.120 INFO:tasks.rados.rados.0.smithi138.stdout:169: oid 2 version 51 is already newer than 51 2023-06-27T12:38:26.120 INFO:tasks.rados.rados.0.smithi138.stdout:169: oid 2 version 51 is already newer than 51 2023-06-27T12:38:26.120 INFO:tasks.rados.rados.0.smithi138.stdout:169: oid 2 version 51 is already newer than 51 2023-06-27T12:38:26.120 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 51 (ObjNum 106 snap 9 seq_num 106) dirty exists 2023-06-27T12:38:26.121 INFO:tasks.rados.rados.0.smithi138.stdout:169: left oid 2 (ObjNum 106 snap 9 seq_num 106) 2023-06-27T12:38:26.121 INFO:tasks.rados.rados.0.smithi138.stdout:168: finishing write tid 2 to smithi138138488-44 2023-06-27T12:38:26.121 INFO:tasks.rados.rados.0.smithi138.stdout:168: finishing write tid 3 to smithi138138488-44 2023-06-27T12:38:26.121 INFO:tasks.rados.rados.0.smithi138.stdout:172: finishing write tid 1 to smithi138138488-15 2023-06-27T12:38:26.122 INFO:tasks.rados.rados.0.smithi138.stdout:174: seq_num 109 ranges {631963=631083,2061949=560019,3243812=504767} 2023-06-27T12:38:26.124 INFO:tasks.rados.rados.0.smithi138.stdout:174: writing smithi138138488-39 from 631963 to 1263046 tid 1 2023-06-27T12:38:26.132 INFO:tasks.rados.rados.0.smithi138.stdout:174: writing smithi138138488-39 from 2061949 to 2621968 tid 2 2023-06-27T12:38:26.140 INFO:tasks.rados.rados.0.smithi138.stdout:174: writing smithi138138488-39 from 3243812 to 3748579 tid 3 2023-06-27T12:38:26.141 INFO:tasks.rados.rados.0.smithi138.stdout:169: done (6 left) 2023-06-27T12:38:26.141 INFO:tasks.rados.rados.0.smithi138.stdout:170: done (5 left) 2023-06-27T12:38:26.141 INFO:tasks.rados.rados.0.smithi138.stdout:173: done (4 left) 2023-06-27T12:38:26.141 INFO:tasks.rados.rados.0.smithi138.stdout:175: read oid 3 snap -1 2023-06-27T12:38:26.141 INFO:tasks.rados.rados.0.smithi138.stdout:175: expect (ObjNum 91 snap 7 seq_num 91) 2023-06-27T12:38:26.141 INFO:tasks.rados.rados.0.smithi138.stdout:176: write oid 14 current snap is 9 2023-06-27T12:38:26.141 INFO:tasks.rados.rados.0.smithi138.stdout:171: finishing write tid 1 to smithi138138488-17 2023-06-27T12:38:26.142 INFO:tasks.rados.rados.0.smithi138.stdout:171: finishing write tid 2 to smithi138138488-17 2023-06-27T12:38:26.142 INFO:tasks.rados.rados.0.smithi138.stdout:176: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2023-06-27T12:38:26.148 INFO:tasks.rados.rados.0.smithi138.stdout:176: writing smithi138138488-14 from 648770 to 1355102 tid 1 2023-06-27T12:38:26.155 INFO:tasks.rados.rados.0.smithi138.stdout:176: writing smithi138138488-14 from 2120431 to 2540461 tid 2 2023-06-27T12:38:26.163 INFO:tasks.rados.rados.0.smithi138.stdout:176: writing smithi138138488-14 from 3271235 to 3703627 tid 3 2023-06-27T12:38:26.163 INFO:tasks.rados.rados.0.smithi138.stdout:177: snap_create 2023-06-27T12:38:26.163 INFO:tasks.rados.rados.0.smithi138.stdout:171: finishing write tid 3 to smithi138138488-17 2023-06-27T12:38:26.163 INFO:tasks.rados.rados.0.smithi138.stdout:171: finishing write tid 4 to smithi138138488-17 2023-06-27T12:38:26.163 INFO:tasks.rados.rados.0.smithi138.stdout:171: finishing write tid 5 to smithi138138488-17 2023-06-27T12:38:26.163 INFO:tasks.rados.rados.0.smithi138.stdout:171: oid 17 updating version 0 to 29 2023-06-27T12:38:26.164 INFO:tasks.rados.rados.0.smithi138.stdout:171: oid 17 version 29 is already newer than 29 2023-06-27T12:38:26.164 INFO:tasks.rados.rados.0.smithi138.stdout:171: oid 17 version 29 is already newer than 29 2023-06-27T12:38:26.164 INFO:tasks.rados.rados.0.smithi138.stdout:171: oid 17 version 29 is already newer than 29 2023-06-27T12:38:26.164 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 29 (ObjNum 107 snap 9 seq_num 107) dirty exists 2023-06-27T12:38:26.164 INFO:tasks.rados.rados.0.smithi138.stdout:171: left oid 17 (ObjNum 107 snap 9 seq_num 107) 2023-06-27T12:38:26.164 INFO:tasks.rados.rados.0.smithi138.stdout:168: finishing write tid 4 to smithi138138488-44 2023-06-27T12:38:26.164 INFO:tasks.rados.rados.0.smithi138.stdout:168: finishing write tid 5 to smithi138138488-44 2023-06-27T12:38:26.165 INFO:tasks.rados.rados.0.smithi138.stdout:168: oid 44 updating version 0 to 44 2023-06-27T12:38:26.165 INFO:tasks.rados.rados.0.smithi138.stdout:168: oid 44 version 44 is already newer than 44 2023-06-27T12:38:26.165 INFO:tasks.rados.rados.0.smithi138.stdout:168: oid 44 version 44 is already newer than 41 2023-06-27T12:38:26.165 INFO:tasks.rados.rados.0.smithi138.stdout:168: oid 44 version 44 is already newer than 44 2023-06-27T12:38:26.165 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 44 (ObjNum 105 snap 9 seq_num 105) dirty exists 2023-06-27T12:38:26.165 INFO:tasks.rados.rados.0.smithi138.stdout:168: left oid 44 (ObjNum 105 snap 9 seq_num 105) 2023-06-27T12:38:26.165 INFO:tasks.rados.rados.0.smithi138.stdout:172: finishing write tid 2 to smithi138138488-15 2023-06-27T12:38:26.165 INFO:tasks.rados.rados.0.smithi138.stdout:172: finishing write tid 3 to smithi138138488-15 2023-06-27T12:38:26.166 INFO:tasks.rados.rados.0.smithi138.stdout:172: finishing write tid 4 to smithi138138488-15 2023-06-27T12:38:26.166 INFO:tasks.rados.rados.0.smithi138.stdout:172: finishing write tid 5 to smithi138138488-15 2023-06-27T12:38:26.166 INFO:tasks.rados.rados.0.smithi138.stdout:172: oid 15 updating version 0 to 10 2023-06-27T12:38:26.166 INFO:tasks.rados.rados.0.smithi138.stdout:172: oid 15 version 10 is already newer than 7 2023-06-27T12:38:26.166 INFO:tasks.rados.rados.0.smithi138.stdout:172: oid 15 version 10 is already newer than 10 2023-06-27T12:38:26.166 INFO:tasks.rados.rados.0.smithi138.stdout:172: oid 15 version 10 is already newer than 10 2023-06-27T12:38:26.166 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 10 (ObjNum 108 snap 9 seq_num 108) dirty exists 2023-06-27T12:38:26.166 INFO:tasks.rados.rados.0.smithi138.stdout:172: left oid 15 (ObjNum 108 snap 9 seq_num 108) 2023-06-27T12:38:26.167 INFO:tasks.rados.rados.0.smithi138.stdout:175: expect (ObjNum 91 snap 7 seq_num 91) 2023-06-27T12:38:26.351 INFO:tasks.rados.rados.0.smithi138.stdout:174: finishing write tid 1 to smithi138138488-39 2023-06-27T12:38:26.351 INFO:tasks.rados.rados.0.smithi138.stdout:176: finishing write tid 1 to smithi138138488-14 2023-06-27T12:38:26.351 INFO:tasks.rados.rados.0.smithi138.stdout:176: finishing write tid 2 to smithi138138488-14 2023-06-27T12:38:26.351 INFO:tasks.rados.rados.0.smithi138.stdout:176: finishing write tid 3 to smithi138138488-14 2023-06-27T12:38:26.351 INFO:tasks.rados.rados.0.smithi138.stdout:176: finishing write tid 4 to smithi138138488-14 2023-06-27T12:38:26.352 INFO:tasks.rados.rados.0.smithi138.stdout:176: finishing write tid 5 to smithi138138488-14 2023-06-27T12:38:26.352 INFO:tasks.rados.rados.0.smithi138.stdout:176: oid 14 updating version 0 to 44 2023-06-27T12:38:26.352 INFO:tasks.rados.rados.0.smithi138.stdout:176: oid 14 version 44 is already newer than 44 2023-06-27T12:38:26.352 INFO:tasks.rados.rados.0.smithi138.stdout:176: oid 14 version 44 is already newer than 44 2023-06-27T12:38:26.352 INFO:tasks.rados.rados.0.smithi138.stdout:176: oid 14 version 44 is already newer than 44 2023-06-27T12:38:26.352 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 44 (ObjNum 110 snap 9 seq_num 110) dirty exists 2023-06-27T12:38:26.352 INFO:tasks.rados.rados.0.smithi138.stdout:176: left oid 14 (ObjNum 110 snap 9 seq_num 110) 2023-06-27T12:38:26.353 INFO:tasks.rados.rados.0.smithi138.stdout:174: finishing write tid 2 to smithi138138488-39 2023-06-27T12:38:26.353 INFO:tasks.rados.rados.0.smithi138.stdout:174: finishing write tid 3 to smithi138138488-39 2023-06-27T12:38:26.353 INFO:tasks.rados.rados.0.smithi138.stdout:174: finishing write tid 4 to smithi138138488-39 2023-06-27T12:38:26.353 INFO:tasks.rados.rados.0.smithi138.stdout:174: finishing write tid 5 to smithi138138488-39 2023-06-27T12:38:26.353 INFO:tasks.rados.rados.0.smithi138.stdout:174: oid 39 updating version 0 to 21 2023-06-27T12:38:26.353 INFO:tasks.rados.rados.0.smithi138.stdout:174: oid 39 updating version 21 to 24 2023-06-27T12:38:26.353 INFO:tasks.rados.rados.0.smithi138.stdout:174: oid 39 version 24 is already newer than 24 2023-06-27T12:38:26.353 INFO:tasks.rados.rados.0.smithi138.stdout:174: oid 39 version 24 is already newer than 24 2023-06-27T12:38:26.354 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 39 v 24 (ObjNum 109 snap 9 seq_num 109) dirty exists 2023-06-27T12:38:26.354 INFO:tasks.rados.rados.0.smithi138.stdout:174: left oid 39 (ObjNum 109 snap 9 seq_num 109) 2023-06-27T12:38:26.476 INFO:tasks.rados.rados.0.smithi138.stdout:168: done (6 left) 2023-06-27T12:38:26.476 INFO:tasks.rados.rados.0.smithi138.stdout:171: done (5 left) 2023-06-27T12:38:26.476 INFO:tasks.rados.rados.0.smithi138.stdout:172: done (4 left) 2023-06-27T12:38:26.476 INFO:tasks.rados.rados.0.smithi138.stdout:174: done (3 left) 2023-06-27T12:38:26.477 INFO:tasks.rados.rados.0.smithi138.stdout:175: done (2 left) 2023-06-27T12:38:26.477 INFO:tasks.rados.rados.0.smithi138.stdout:176: done (1 left) 2023-06-27T12:38:26.477 INFO:tasks.rados.rados.0.smithi138.stdout:177: done (0 left) 2023-06-27T12:38:26.477 INFO:tasks.rados.rados.0.smithi138.stdout:178: delete oid 23 current snap is 10 2023-06-27T12:38:26.517 INFO:tasks.rados.rados.0.smithi138.stdout:178: done (0 left) 2023-06-27T12:38:26.517 INFO:tasks.rados.rados.0.smithi138.stdout:179: delete oid 36 current snap is 10 2023-06-27T12:38:26.575 INFO:tasks.rados.rados.0.smithi138.stdout:179: done (0 left) 2023-06-27T12:38:26.575 INFO:tasks.rados.rados.0.smithi138.stdout:180: write oid 41 current snap is 10 2023-06-27T12:38:26.576 INFO:tasks.rados.rados.0.smithi138.stdout:180: seq_num 113 ranges {699191=532079,1779524=567357,2837151=666208} 2023-06-27T12:38:26.582 INFO:tasks.rados.rados.0.smithi138.stdout:180: writing smithi138138488-41 from 699191 to 1231270 tid 1 2023-06-27T12:38:26.588 INFO:tasks.rados.rados.0.smithi138.stdout:180: writing smithi138138488-41 from 1779524 to 2346881 tid 2 2023-06-27T12:38:26.594 INFO:tasks.rados.rados.0.smithi138.stdout:180: writing smithi138138488-41 from 2837151 to 3503359 tid 3 2023-06-27T12:38:26.594 INFO:tasks.rados.rados.0.smithi138.stdout:181: write oid 20 current snap is 10 2023-06-27T12:38:26.595 INFO:tasks.rados.rados.0.smithi138.stdout:181: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2023-06-27T12:38:26.601 INFO:tasks.rados.rados.0.smithi138.stdout:181: writing smithi138138488-20 from 715998 to 1323326 tid 1 2023-06-27T12:38:26.607 INFO:tasks.rados.rados.0.smithi138.stdout:181: writing smithi138138488-20 from 1838006 to 2265374 tid 2 2023-06-27T12:38:26.614 INFO:tasks.rados.rados.0.smithi138.stdout:181: writing smithi138138488-20 from 2864574 to 3342054 tid 3 2023-06-27T12:38:26.617 INFO:tasks.rados.rados.0.smithi138.stdout:181: writing smithi138138488-20 from 3863835 to 3915998 tid 4 2023-06-27T12:38:26.617 INFO:tasks.rados.rados.0.smithi138.stdout:182: snap_remove snap 9 2023-06-27T12:38:27.252 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:27.380 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:27.375+0000 7ff404cc2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:27.383 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:27.379+0000 7ff404cc2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:27.477 INFO:tasks.rados.rados.0.smithi138.stdout:182: done (2 left) 2023-06-27T12:38:27.478 INFO:tasks.rados.rados.0.smithi138.stdout:183: delete oid 30 current snap is 10 2023-06-27T12:38:27.478 INFO:tasks.rados.rados.0.smithi138.stdout:180: finishing write tid 1 to smithi138138488-41 2023-06-27T12:38:27.478 INFO:tasks.rados.rados.0.smithi138.stdout:180: finishing write tid 2 to smithi138138488-41 2023-06-27T12:38:27.478 INFO:tasks.rados.rados.0.smithi138.stdout:180: finishing write tid 3 to smithi138138488-41 2023-06-27T12:38:27.478 INFO:tasks.rados.rados.0.smithi138.stdout:180: finishing write tid 4 to smithi138138488-41 2023-06-27T12:38:27.478 INFO:tasks.rados.rados.0.smithi138.stdout:180: finishing write tid 5 to smithi138138488-41 2023-06-27T12:38:27.479 INFO:tasks.rados.rados.0.smithi138.stdout:180: oid 41 updating version 0 to 40 2023-06-27T12:38:27.479 INFO:tasks.rados.rados.0.smithi138.stdout:180: oid 41 version 40 is already newer than 40 2023-06-27T12:38:27.479 INFO:tasks.rados.rados.0.smithi138.stdout:180: oid 41 version 40 is already newer than 40 2023-06-27T12:38:27.479 INFO:tasks.rados.rados.0.smithi138.stdout:180: oid 41 version 40 is already newer than 40 2023-06-27T12:38:27.479 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 41 v 40 (ObjNum 113 snap 10 seq_num 113) dirty exists 2023-06-27T12:38:27.479 INFO:tasks.rados.rados.0.smithi138.stdout:180: left oid 41 (ObjNum 113 snap 10 seq_num 113) 2023-06-27T12:38:27.479 INFO:tasks.rados.rados.0.smithi138.stdout:181: finishing write tid 1 to smithi138138488-20 2023-06-27T12:38:27.479 INFO:tasks.rados.rados.0.smithi138.stdout:181: finishing write tid 2 to smithi138138488-20 2023-06-27T12:38:27.480 INFO:tasks.rados.rados.0.smithi138.stdout:181: finishing write tid 3 to smithi138138488-20 2023-06-27T12:38:27.480 INFO:tasks.rados.rados.0.smithi138.stdout:181: finishing write tid 4 to smithi138138488-20 2023-06-27T12:38:27.480 INFO:tasks.rados.rados.0.smithi138.stdout:181: finishing write tid 5 to smithi138138488-20 2023-06-27T12:38:27.480 INFO:tasks.rados.rados.0.smithi138.stdout:181: finishing write tid 6 to smithi138138488-20 2023-06-27T12:38:27.480 INFO:tasks.rados.rados.0.smithi138.stdout:181: oid 20 updating version 0 to 35 2023-06-27T12:38:27.480 INFO:tasks.rados.rados.0.smithi138.stdout:181: oid 20 version 35 is already newer than 35 2023-06-27T12:38:27.480 INFO:tasks.rados.rados.0.smithi138.stdout:181: oid 20 version 35 is already newer than 35 2023-06-27T12:38:27.481 INFO:tasks.rados.rados.0.smithi138.stdout:181: oid 20 version 35 is already newer than 35 2023-06-27T12:38:27.481 INFO:tasks.rados.rados.0.smithi138.stdout:181: oid 20 version 35 is already newer than 35 2023-06-27T12:38:27.481 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 35 (ObjNum 114 snap 10 seq_num 114) dirty exists 2023-06-27T12:38:27.481 INFO:tasks.rados.rados.0.smithi138.stdout:181: left oid 20 (ObjNum 114 snap 10 seq_num 114) 2023-06-27T12:38:27.540 INFO:tasks.rados.rados.0.smithi138.stdout:180: done (2 left) 2023-06-27T12:38:27.540 INFO:tasks.rados.rados.0.smithi138.stdout:181: done (1 left) 2023-06-27T12:38:27.540 INFO:tasks.rados.rados.0.smithi138.stdout:183: done (0 left) 2023-06-27T12:38:27.540 INFO:tasks.rados.rados.0.smithi138.stdout:184: write (excl) oid 45 current snap is 10 2023-06-27T12:38:27.540 INFO:tasks.rados.rados.0.smithi138.stdout:184: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2023-06-27T12:38:27.546 INFO:tasks.rados.rados.0.smithi138.stdout:184: writing smithi138138488-45 from 749612 to 1223791 tid 1 2023-06-27T12:38:27.552 INFO:tasks.rados.rados.0.smithi138.stdout:184: writing smithi138138488-45 from 1954970 to 2386007 tid 2 2023-06-27T12:38:27.558 INFO:tasks.rados.rados.0.smithi138.stdout:184: writing smithi138138488-45 from 2919420 to 3535797 tid 3 2023-06-27T12:38:27.559 INFO:tasks.rados.rados.0.smithi138.stdout:185: write (excl) oid 4 current snap is 10 2023-06-27T12:38:27.559 INFO:tasks.rados.rados.0.smithi138.stdout:185: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2023-06-27T12:38:27.566 INFO:tasks.rados.rados.0.smithi138.stdout:185: writing smithi138138488-4 from 766419 to 1315847 tid 1 2023-06-27T12:38:27.573 INFO:tasks.rados.rados.0.smithi138.stdout:185: writing smithi138138488-4 from 2013452 to 2588147 tid 2 2023-06-27T12:38:27.580 INFO:tasks.rados.rados.0.smithi138.stdout:185: writing smithi138138488-4 from 3346843 to 3774492 tid 3 2023-06-27T12:38:27.581 INFO:tasks.rados.rados.0.smithi138.stdout:186: delete oid 30 current snap is 10 2023-06-27T12:38:27.581 INFO:tasks.rados.rados.0.smithi138.stdout:184: finishing write tid 1 to smithi138138488-45 2023-06-27T12:38:27.603 INFO:tasks.rados.rados.0.smithi138.stdout:184: finishing write tid 2 to smithi138138488-45 2023-06-27T12:38:27.644 INFO:tasks.rados.rados.0.smithi138.stdout:184: finishing write tid 3 to smithi138138488-45 2023-06-27T12:38:27.644 INFO:tasks.rados.rados.0.smithi138.stdout:184: finishing write tid 4 to smithi138138488-45 2023-06-27T12:38:27.644 INFO:tasks.rados.rados.0.smithi138.stdout:184: finishing write tid 5 to smithi138138488-45 2023-06-27T12:38:27.644 INFO:tasks.rados.rados.0.smithi138.stdout:184: oid 45 updating version 0 to 55 2023-06-27T12:38:27.644 INFO:tasks.rados.rados.0.smithi138.stdout:184: oid 45 version 55 is already newer than 55 2023-06-27T12:38:27.644 INFO:tasks.rados.rados.0.smithi138.stdout:184: oid 45 version 55 is already newer than 55 2023-06-27T12:38:27.645 INFO:tasks.rados.rados.0.smithi138.stdout:184: oid 45 version 55 is already newer than 52 2023-06-27T12:38:27.645 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 55 (ObjNum 116 snap 10 seq_num 116) dirty exists 2023-06-27T12:38:27.645 INFO:tasks.rados.rados.0.smithi138.stdout:184: left oid 45 (ObjNum 116 snap 10 seq_num 116) 2023-06-27T12:38:27.645 INFO:tasks.rados.rados.0.smithi138.stdout:185: finishing write tid 1 to smithi138138488-4 2023-06-27T12:38:27.667 INFO:tasks.rados.rados.0.smithi138.stdout:185: finishing write tid 2 to smithi138138488-4 2023-06-27T12:38:27.667 INFO:tasks.rados.rados.0.smithi138.stdout:185: finishing write tid 3 to smithi138138488-4 2023-06-27T12:38:27.667 INFO:tasks.rados.rados.0.smithi138.stdout:185: finishing write tid 4 to smithi138138488-4 2023-06-27T12:38:27.669 INFO:tasks.rados.rados.0.smithi138.stdout:185: finishing write tid 5 to smithi138138488-4 2023-06-27T12:38:27.669 INFO:tasks.rados.rados.0.smithi138.stdout:185: oid 4 updating version 0 to 29 2023-06-27T12:38:27.669 INFO:tasks.rados.rados.0.smithi138.stdout:185: oid 4 version 29 is already newer than 29 2023-06-27T12:38:27.669 INFO:tasks.rados.rados.0.smithi138.stdout:185: oid 4 version 29 is already newer than 29 2023-06-27T12:38:27.669 INFO:tasks.rados.rados.0.smithi138.stdout:185: oid 4 version 29 is already newer than 26 2023-06-27T12:38:27.669 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 29 (ObjNum 117 snap 10 seq_num 117) dirty exists 2023-06-27T12:38:27.669 INFO:tasks.rados.rados.0.smithi138.stdout:185: left oid 4 (ObjNum 117 snap 10 seq_num 117) 2023-06-27T12:38:27.670 INFO:tasks.rados.rados.0.smithi138.stdout:184: done (2 left) 2023-06-27T12:38:27.670 INFO:tasks.rados.rados.0.smithi138.stdout:185: done (1 left) 2023-06-27T12:38:27.670 INFO:tasks.rados.rados.0.smithi138.stdout:186: done (0 left) 2023-06-27T12:38:27.670 INFO:tasks.rados.rados.0.smithi138.stdout:187: write (excl) oid 46 current snap is 10 2023-06-27T12:38:27.670 INFO:tasks.rados.rados.0.smithi138.stdout:187: seq_num 119 ranges {400033=699926,1614063=385970} 2023-06-27T12:38:27.674 INFO:tasks.rados.rados.0.smithi138.stdout:187: writing smithi138138488-46 from 400033 to 1099959 tid 1 2023-06-27T12:38:27.679 INFO:tasks.rados.rados.0.smithi138.stdout:187: writing smithi138138488-46 from 1614063 to 2000033 tid 2 2023-06-27T12:38:27.680 INFO:tasks.rados.rados.0.smithi138.stdout:188: write oid 8 current snap is 10 2023-06-27T12:38:27.680 INFO:tasks.rados.rados.0.smithi138.stdout:188: seq_num 120 ranges {416840=775175,1672545=344295} 2023-06-27T12:38:27.686 INFO:tasks.rados.rados.0.smithi138.stdout:188: writing smithi138138488-8 from 416840 to 1192015 tid 1 2023-06-27T12:38:27.691 INFO:tasks.rados.rados.0.smithi138.stdout:188: writing smithi138138488-8 from 1672545 to 2016840 tid 2 2023-06-27T12:38:27.691 INFO:tasks.rados.rados.0.smithi138.stdout:189: snap_create 2023-06-27T12:38:27.762 INFO:tasks.rados.rados.0.smithi138.stdout:188: finishing write tid 1 to smithi138138488-8 2023-06-27T12:38:27.767 INFO:tasks.rados.rados.0.smithi138.stdout:187: finishing write tid 1 to smithi138138488-46 2023-06-27T12:38:27.792 INFO:tasks.rados.rados.0.smithi138.stdout:187: finishing write tid 2 to smithi138138488-46 2023-06-27T12:38:27.793 INFO:tasks.rados.rados.0.smithi138.stdout:187: finishing write tid 3 to smithi138138488-46 2023-06-27T12:38:27.793 INFO:tasks.rados.rados.0.smithi138.stdout:187: finishing write tid 4 to smithi138138488-46 2023-06-27T12:38:27.794 INFO:tasks.rados.rados.0.smithi138.stdout:187: oid 46 updating version 0 to 27 2023-06-27T12:38:27.794 INFO:tasks.rados.rados.0.smithi138.stdout:187: oid 46 version 27 is already newer than 25 2023-06-27T12:38:27.794 INFO:tasks.rados.rados.0.smithi138.stdout:187: oid 46 version 27 is already newer than 27 2023-06-27T12:38:27.794 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 27 (ObjNum 119 snap 10 seq_num 119) dirty exists 2023-06-27T12:38:27.794 INFO:tasks.rados.rados.0.smithi138.stdout:187: left oid 46 (ObjNum 119 snap 10 seq_num 119) 2023-06-27T12:38:27.802 INFO:tasks.rados.rados.0.smithi138.stdout:188: finishing write tid 2 to smithi138138488-8 2023-06-27T12:38:27.803 INFO:tasks.rados.rados.0.smithi138.stdout:188: finishing write tid 3 to smithi138138488-8 2023-06-27T12:38:27.803 INFO:tasks.rados.rados.0.smithi138.stdout:188: finishing write tid 4 to smithi138138488-8 2023-06-27T12:38:27.803 INFO:tasks.rados.rados.0.smithi138.stdout:188: oid 8 updating version 0 to 13 2023-06-27T12:38:27.803 INFO:tasks.rados.rados.0.smithi138.stdout:188: oid 8 version 13 is already newer than 13 2023-06-27T12:38:27.803 INFO:tasks.rados.rados.0.smithi138.stdout:188: oid 8 version 13 is already newer than 13 2023-06-27T12:38:27.803 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 8 v 13 (ObjNum 120 snap 10 seq_num 120) dirty exists 2023-06-27T12:38:27.803 INFO:tasks.rados.rados.0.smithi138.stdout:188: left oid 8 (ObjNum 120 snap 10 seq_num 120) 2023-06-27T12:38:28.480 INFO:tasks.rados.rados.0.smithi138.stdout:187: done (2 left) 2023-06-27T12:38:28.481 INFO:tasks.rados.rados.0.smithi138.stdout:188: done (1 left) 2023-06-27T12:38:28.481 INFO:tasks.rados.rados.0.smithi138.stdout:189: done (0 left) 2023-06-27T12:38:28.481 INFO:tasks.rados.rados.0.smithi138.stdout:190: snap_create 2023-06-27T12:38:29.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:29.482 INFO:tasks.rados.rados.0.smithi138.stdout:190: done (0 left) 2023-06-27T12:38:29.483 INFO:tasks.rados.rados.0.smithi138.stdout:191: read oid 2 snap -1 2023-06-27T12:38:29.483 INFO:tasks.rados.rados.0.smithi138.stdout:191: expect (ObjNum 106 snap 9 seq_num 106) 2023-06-27T12:38:29.483 INFO:tasks.rados.rados.0.smithi138.stdout:192: snap_create 2023-06-27T12:38:29.535 INFO:tasks.rados.rados.0.smithi138.stdout:191: expect (ObjNum 106 snap 9 seq_num 106) 2023-06-27T12:38:29.549 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:29.550 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:29.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:29.601 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:29.595+0000 7fd07b2e9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:29.604 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:29.600+0000 7fd07b2e9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:29.698 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:29.691+0000 7f8b77a23700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:29.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:29.697+0000 7f8b77a23700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:29.903 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:29.904 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:29.917 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2023-06-27T12:38:29.477500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":130960980,"num_objects":69,"num_object_clones":21,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":451,"num_read_kb":195708,"num_write":583,"num_write_kb":131273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":524,"ondisk_log_size":524,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":354968,"kb_used_data":240476,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377132392,"statfs":{"total":386547056640,"available":386183569408,"internally_reserved":0,"allocated":246247424,"data_stored":249219711,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1714116,"num_objects":-1,"num_object_clones":2,"num_object_copies":-3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":45,"num_read_kb":23806,"num_write":64,"num_write_kb":20314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013697"},"pg_stats":[{"pgid":"3.f","version":"18'4","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.551694+0000","last_change":"2023-06-27T12:38:27.799645+0000","last_active":"2023-06-27T12:38:28.551694+0000","last_peered":"2023-06-27T12:38:28.551694+0000","last_clean":"2023-06-27T12:38:28.551694+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:28.551694+0000","last_undegraded":"2023-06-27T12:38:28.551694+0000","last_fullsized":"2023-06-27T12:38:28.551694+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"38'35","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.486574+0000","last_change":"2023-06-27T12:38:27.788425+0000","last_active":"2023-06-27T12:38:28.486574+0000","last_peered":"2023-06-27T12:38:28.486574+0000","last_clean":"2023-06-27T12:38:28.486574+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:28.486574+0000","last_undegraded":"2023-06-27T12:38:28.486574+0000","last_fullsized":"2023-06-27T12:38:28.486574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13680766,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":17650,"num_write":36,"num_write_kb":10378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"38'31","reported_seq":101,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.526230+0000","last_change":"2023-06-27T12:38:27.486169+0000","last_active":"2023-06-27T12:38:28.526230+0000","last_peered":"2023-06-27T12:38:28.526230+0000","last_clean":"2023-06-27T12:38:28.526230+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:28.526230+0000","last_undegraded":"2023-06-27T12:38:28.526230+0000","last_fullsized":"2023-06-27T12:38:28.526230+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5831963,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":12452,"num_write":35,"num_write_kb":9601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"32'22","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.550600+0000","last_change":"2023-06-27T12:38:27.809325+0000","last_active":"2023-06-27T12:38:28.550600+0000","last_peered":"2023-06-27T12:38:28.550600+0000","last_clean":"2023-06-27T12:38:28.550600+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:28.550600+0000","last_undegraded":"2023-06-27T12:38:28.550600+0000","last_fullsized":"2023-06-27T12:38:28.550600+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9025260,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":18,"num_read_kb":3227,"num_write":22,"num_write_kb":6297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"39'13","reported_seq":80,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.555648+0000","last_change":"2023-06-27T12:38:27.492501+0000","last_active":"2023-06-27T12:38:28.555648+0000","last_peered":"2023-06-27T12:38:28.555648+0000","last_clean":"2023-06-27T12:38:28.555648+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:28.555648+0000","last_undegraded":"2023-06-27T12:38:28.555648+0000","last_fullsized":"2023-06-27T12:38:28.555648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5831996,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11425,"num_write":17,"num_write_kb":4485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"37'44","reported_seq":111,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.552601+0000","last_change":"2023-06-27T12:38:27.804009+0000","last_active":"2023-06-27T12:38:28.552601+0000","last_peered":"2023-06-27T12:38:28.552601+0000","last_clean":"2023-06-27T12:38:28.552601+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:28.552601+0000","last_undegraded":"2023-06-27T12:38:28.552601+0000","last_fullsized":"2023-06-27T12:38:28.552601+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327819,"num_objects":8,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":16659,"num_write":45,"num_write_kb":12861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"31'31","reported_seq":95,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.552551+0000","last_change":"2023-06-27T12:38:27.808911+0000","last_active":"2023-06-27T12:38:28.552551+0000","last_peered":"2023-06-27T12:38:28.552551+0000","last_clean":"2023-06-27T12:38:28.552551+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:28.552551+0000","last_undegraded":"2023-06-27T12:38:28.552551+0000","last_fullsized":"2023-06-27T12:38:28.552551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8739541,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17108,"num_write":30,"num_write_kb":8934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"28'8","reported_seq":74,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.532363+0000","last_change":"2023-06-27T12:38:27.485947+0000","last_active":"2023-06-27T12:38:28.532363+0000","last_peered":"2023-06-27T12:38:28.532363+0000","last_clean":"2023-06-27T12:38:28.532363+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:28.532363+0000","last_undegraded":"2023-06-27T12:38:28.532363+0000","last_fullsized":"2023-06-27T12:38:28.532363+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3327753,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":11710,"num_write":9,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.487944+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:28.487944+0000","last_peered":"2023-06-27T12:38:28.487944+0000","last_clean":"2023-06-27T12:38:28.487944+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:28.487944+0000","last_undegraded":"2023-06-27T12:38:28.487944+0000","last_fullsized":"2023-06-27T12:38:28.487944+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.527674+0000","last_change":"2023-06-27T12:38:27.486595+0000","last_active":"2023-06-27T12:38:28.527674+0000","last_peered":"2023-06-27T12:38:28.527674+0000","last_clean":"2023-06-27T12:38:28.527674+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:28.527674+0000","last_undegraded":"2023-06-27T12:38:28.527674+0000","last_fullsized":"2023-06-27T12:38:28.527674+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.552341+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:28.552341+0000","last_peered":"2023-06-27T12:38:28.552341+0000","last_clean":"2023-06-27T12:38:28.552341+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:28.552341+0000","last_undegraded":"2023-06-27T12:38:28.552341+0000","last_fullsized":"2023-06-27T12:38:28.552341+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"39'27","reported_seq":90,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.489248+0000","last_change":"2023-06-27T12:38:27.708189+0000","last_active":"2023-06-27T12:38:28.489248+0000","last_peered":"2023-06-27T12:38:28.489248+0000","last_clean":"2023-06-27T12:38:28.489248+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:28.489248+0000","last_undegraded":"2023-06-27T12:38:28.489248+0000","last_fullsized":"2023-06-27T12:38:28.489248+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5109295,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":2666,"num_write":25,"num_write_kb":7014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.524381+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:28.524381+0000","last_peered":"2023-06-27T12:38:28.524381+0000","last_clean":"2023-06-27T12:38:28.524381+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:28.524381+0000","last_undegraded":"2023-06-27T12:38:28.524381+0000","last_fullsized":"2023-06-27T12:38:28.524381+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"39'55","reported_seq":137,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.555051+0000","last_change":"2023-06-27T12:38:27.491626+0000","last_active":"2023-06-27T12:38:28.555051+0000","last_peered":"2023-06-27T12:38:28.555051+0000","last_clean":"2023-06-27T12:38:28.555051+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:28.555051+0000","last_undegraded":"2023-06-27T12:38:28.555051+0000","last_fullsized":"2023-06-27T12:38:28.555051+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17227010,"num_objects":7,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":31225,"num_write":63,"num_write_kb":18831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.524299+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:28.524299+0000","last_peered":"2023-06-27T12:38:28.524299+0000","last_clean":"2023-06-27T12:38:28.524299+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:28.524299+0000","last_undegraded":"2023-06-27T12:38:28.524299+0000","last_fullsized":"2023-06-27T12:38:28.524299+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":182,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.558827+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:28.558827+0000","last_peered":"2023-06-27T12:38:28.558827+0000","last_clean":"2023-06-27T12:38:28.558827+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:28.558827+0000","last_undegraded":"2023-06-27T12:38:28.558827+0000","last_fullsized":"2023-06-27T12:38:28.558827+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"37'44","reported_seq":119,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.529117+0000","last_change":"2023-06-27T12:38:27.485545+0000","last_active":"2023-06-27T12:38:28.529117+0000","last_peered":"2023-06-27T12:38:28.529117+0000","last_clean":"2023-06-27T12:38:28.529117+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:28.529117+0000","last_undegraded":"2023-06-27T12:38:28.529117+0000","last_fullsized":"2023-06-27T12:38:28.529117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15579924,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":48,"num_read_kb":19114,"num_write":49,"num_write_kb":13644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.550698+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:28.550698+0000","last_peered":"2023-06-27T12:38:28.550698+0000","last_clean":"2023-06-27T12:38:28.550698+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:28.550698+0000","last_undegraded":"2023-06-27T12:38:28.550698+0000","last_fullsized":"2023-06-27T12:38:28.550698+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"39'29","reported_seq":99,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.555580+0000","last_change":"2023-06-27T12:38:27.492375+0000","last_active":"2023-06-27T12:38:28.555580+0000","last_peered":"2023-06-27T12:38:28.555580+0000","last_clean":"2023-06-27T12:38:28.555580+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:28.555580+0000","last_undegraded":"2023-06-27T12:38:28.555580+0000","last_fullsized":"2023-06-27T12:38:28.555580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798382,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.559793+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:28.559793+0000","last_peered":"2023-06-27T12:38:28.559793+0000","last_clean":"2023-06-27T12:38:28.559793+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:28.559793+0000","last_undegraded":"2023-06-27T12:38:28.559793+0000","last_fullsized":"2023-06-27T12:38:28.559793+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"38'40","reported_seq":111,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.487271+0000","last_change":"2023-06-27T12:38:27.759301+0000","last_active":"2023-06-27T12:38:28.487271+0000","last_peered":"2023-06-27T12:38:28.487271+0000","last_clean":"2023-06-27T12:38:28.487271+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:28.487271+0000","last_undegraded":"2023-06-27T12:38:28.487271+0000","last_fullsized":"2023-06-27T12:38:28.487271+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13832029,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":15303,"num_write":41,"num_write_kb":12814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"33'10","reported_seq":72,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.554907+0000","last_change":"2023-06-27T12:38:27.492088+0000","last_active":"2023-06-27T12:38:28.554907+0000","last_peered":"2023-06-27T12:38:28.554907+0000","last_clean":"2023-06-27T12:38:28.554907+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:28.554907+0000","last_undegraded":"2023-06-27T12:38:28.554907+0000","last_fullsized":"2023-06-27T12:38:28.554907+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5915998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":3466,"num_write":13,"num_write_kb":3651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.488855+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:28.488855+0000","last_peered":"2023-06-27T12:38:28.488855+0000","last_clean":"2023-06-27T12:38:28.488855+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:28.488855+0000","last_undegraded":"2023-06-27T12:38:28.488855+0000","last_fullsized":"2023-06-27T12:38:28.488855+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.531136+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:28.531136+0000","last_peered":"2023-06-27T12:38:28.531136+0000","last_clean":"2023-06-27T12:38:28.531136+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:28.531136+0000","last_undegraded":"2023-06-27T12:38:28.531136+0000","last_fullsized":"2023-06-27T12:38:28.531136+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"37'34","reported_seq":101,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-06-27T12:38:28.483938+0000","last_change":"2023-06-27T12:38:27.690391+0000","last_active":"2023-06-27T12:38:28.483938+0000","last_peered":"2023-06-27T12:38:28.483938+0000","last_clean":"2023-06-27T12:38:28.483938+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:28.483938+0000","last_undegraded":"2023-06-27T12:38:28.483938+0000","last_fullsized":"2023-06-27T12:38:28.483938+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3142876,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":20,"num_read_kb":14986,"num_write":27,"num_write_kb":8065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":130370612,"num_objects":67,"num_object_clones":21,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":348,"num_read_kb":195617,"num_write":446,"num_write_kb":129379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":427,"ondisk_log_size":427,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":92276,"kb_used_data":62900,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94279564,"statfs":{"total":96636764160,"available":96542273536,"internally_reserved":0,"allocated":64409600,"data_stored":64674696,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949672995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89480,"kb_used_data":60440,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94282360,"statfs":{"total":96636764160,"available":96545136640,"internally_reserved":0,"allocated":61890560,"data_stored":63040752,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705700,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83536,"kb_used_data":55040,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94288304,"statfs":{"total":96636764160,"available":96551223296,"internally_reserved":0,"allocated":56360960,"data_stored":57286465,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949672995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89676,"kb_used_data":62096,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94282164,"statfs":{"total":96636764160,"available":96544935936,"internally_reserved":0,"allocated":63586304,"data_stored":64217798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:30.488 INFO:tasks.rados.rados.0.smithi138.stdout:191: done (1 left) 2023-06-27T12:38:30.489 INFO:tasks.rados.rados.0.smithi138.stdout:192: done (0 left) 2023-06-27T12:38:30.489 INFO:tasks.rados.rados.0.smithi138.stdout:193: snap_create 2023-06-27T12:38:31.490 INFO:tasks.rados.rados.0.smithi138.stdout:193: done (0 left) 2023-06-27T12:38:31.490 INFO:tasks.rados.rados.0.smithi138.stdout:194: delete oid 46 current snap is 14 2023-06-27T12:38:31.536 INFO:tasks.rados.rados.0.smithi138.stdout:194: done (0 left) 2023-06-27T12:38:31.537 INFO:tasks.rados.rados.0.smithi138.stdout:195: snap_remove snap 13 2023-06-27T12:38:31.690 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:31.826 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:31.822+0000 7fb1a2963700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:31.829 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:31.825+0000 7fb1a2963700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:32.496 INFO:tasks.rados.rados.0.smithi138.stdout:195: done (0 left) 2023-06-27T12:38:32.497 INFO:tasks.rados.rados.0.smithi138.stdout:196: write (excl) oid 40 current snap is 14 2023-06-27T12:38:32.497 INFO:tasks.rados.rados.0.smithi138.stdout:196: seq_num 122 ranges {450454=642026,1505862=544592} 2023-06-27T12:38:32.502 INFO:tasks.rados.rados.0.smithi138.stdout:196: writing smithi138138488-40 from 450454 to 1092480 tid 1 2023-06-27T12:38:32.508 INFO:tasks.rados.rados.0.smithi138.stdout:196: writing smithi138138488-40 from 1505862 to 2050454 tid 2 2023-06-27T12:38:32.508 INFO:tasks.rados.rados.0.smithi138.stdout:197: read oid 8 snap 12 2023-06-27T12:38:32.508 INFO:tasks.rados.rados.0.smithi138.stdout:197: expect (ObjNum 120 snap 10 seq_num 120) 2023-06-27T12:38:32.508 INFO:tasks.rados.rados.0.smithi138.stdout:198: delete oid 17 current snap is 14 2023-06-27T12:38:32.616 INFO:tasks.rados.rados.0.smithi138.stdout:197: expect (ObjNum 120 snap 10 seq_num 120) 2023-06-27T12:38:32.691 INFO:tasks.rados.rados.0.smithi138.stdout:196: finishing write tid 1 to smithi138138488-40 2023-06-27T12:38:32.692 INFO:tasks.rados.rados.0.smithi138.stdout:196: finishing write tid 2 to smithi138138488-40 2023-06-27T12:38:32.692 INFO:tasks.rados.rados.0.smithi138.stdout:196: finishing write tid 3 to smithi138138488-40 2023-06-27T12:38:32.692 INFO:tasks.rados.rados.0.smithi138.stdout:196: finishing write tid 4 to smithi138138488-40 2023-06-27T12:38:32.692 INFO:tasks.rados.rados.0.smithi138.stdout:196: oid 40 updating version 0 to 26 2023-06-27T12:38:32.693 INFO:tasks.rados.rados.0.smithi138.stdout:196: oid 40 version 26 is already newer than 23 2023-06-27T12:38:32.693 INFO:tasks.rados.rados.0.smithi138.stdout:196: oid 40 version 26 is already newer than 26 2023-06-27T12:38:32.693 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 26 (ObjNum 122 snap 14 seq_num 122) dirty exists 2023-06-27T12:38:32.694 INFO:tasks.rados.rados.0.smithi138.stdout:196: left oid 40 (ObjNum 122 snap 14 seq_num 122) 2023-06-27T12:38:32.694 INFO:tasks.rados.rados.0.smithi138.stdout:196: done (2 left) 2023-06-27T12:38:32.694 INFO:tasks.rados.rados.0.smithi138.stdout:197: done (1 left) 2023-06-27T12:38:32.695 INFO:tasks.rados.rados.0.smithi138.stdout:198: done (0 left) 2023-06-27T12:38:32.695 INFO:tasks.rados.rados.0.smithi138.stdout:199: snap_create 2023-06-27T12:38:33.500 INFO:tasks.rados.rados.0.smithi138.stdout:199: done (0 left) 2023-06-27T12:38:33.500 INFO:tasks.rados.rados.0.smithi138.stdout:200: write (excl) oid 33 current snap is 15 2023-06-27T12:38:33.501 INFO:tasks.rados.rados.0.smithi138.stdout:200: seq_num 124 ranges {484068=792524,1906473=177595} 2023-06-27T12:38:33.507 INFO:tasks.rados.rados.0.smithi138.stdout:200: writing smithi138138488-33 from 484068 to 1276592 tid 1 2023-06-27T12:38:33.513 INFO:tasks.rados.rados.0.smithi138.stdout:200: writing smithi138138488-33 from 1906473 to 2084068 tid 2 2023-06-27T12:38:33.513 INFO:tasks.rados.rados.0.smithi138.stdout:201: read oid 46 snap 12 2023-06-27T12:38:33.514 INFO:tasks.rados.rados.0.smithi138.stdout:201: expect (ObjNum 119 snap 10 seq_num 119) 2023-06-27T12:38:33.514 INFO:tasks.rados.rados.0.smithi138.stdout:202: delete oid 19 current snap is 15 2023-06-27T12:38:33.569 INFO:tasks.rados.rados.0.smithi138.stdout:201: expect (ObjNum 119 snap 10 seq_num 119) 2023-06-27T12:38:33.666 INFO:tasks.rados.rados.0.smithi138.stdout:200: finishing write tid 1 to smithi138138488-33 2023-06-27T12:38:33.666 INFO:tasks.rados.rados.0.smithi138.stdout:200: finishing write tid 2 to smithi138138488-33 2023-06-27T12:38:33.666 INFO:tasks.rados.rados.0.smithi138.stdout:200: finishing write tid 3 to smithi138138488-33 2023-06-27T12:38:33.666 INFO:tasks.rados.rados.0.smithi138.stdout:200: finishing write tid 4 to smithi138138488-33 2023-06-27T12:38:33.666 INFO:tasks.rados.rados.0.smithi138.stdout:200: oid 33 updating version 0 to 30 2023-06-27T12:38:33.667 INFO:tasks.rados.rados.0.smithi138.stdout:200: oid 33 version 30 is already newer than 27 2023-06-27T12:38:33.667 INFO:tasks.rados.rados.0.smithi138.stdout:200: oid 33 version 30 is already newer than 30 2023-06-27T12:38:33.667 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 30 (ObjNum 124 snap 15 seq_num 124) dirty exists 2023-06-27T12:38:33.667 INFO:tasks.rados.rados.0.smithi138.stdout:200: left oid 33 (ObjNum 124 snap 15 seq_num 124) 2023-06-27T12:38:33.667 INFO:tasks.rados.rados.0.smithi138.stdout:200: done (2 left) 2023-06-27T12:38:33.667 INFO:tasks.rados.rados.0.smithi138.stdout:201: done (1 left) 2023-06-27T12:38:33.667 INFO:tasks.rados.rados.0.smithi138.stdout:202: done (0 left) 2023-06-27T12:38:33.667 INFO:tasks.rados.rados.0.smithi138.stdout:203: snap_remove snap 14 2023-06-27T12:38:33.900 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:34.023+0000 7f578840f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:34.027+0000 7f578840f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:34.504 INFO:tasks.rados.rados.0.smithi138.stdout:203: done (0 left) 2023-06-27T12:38:34.504 INFO:tasks.rados.rados.0.smithi138.stdout:204: read oid 16 snap -1 2023-06-27T12:38:34.504 INFO:tasks.rados.rados.0.smithi138.stdout:204: expect (ObjNum 15 snap 0 seq_num 15) 2023-06-27T12:38:34.504 INFO:tasks.rados.rados.0.smithi138.stdout:205: write (excl) oid 7 current snap is 15 2023-06-27T12:38:34.504 INFO:tasks.rados.rados.0.smithi138.stdout:205: seq_num 126 ranges {517682=543022,1623437=494245} 2023-06-27T12:38:34.509 INFO:tasks.rados.rados.0.smithi138.stdout:205: writing smithi138138488-7 from 517682 to 1060704 tid 1 2023-06-27T12:38:34.516 INFO:tasks.rados.rados.0.smithi138.stdout:205: writing smithi138138488-7 from 1623437 to 2117682 tid 2 2023-06-27T12:38:34.516 INFO:tasks.rados.rados.0.smithi138.stdout:206: read oid 21 snap -1 2023-06-27T12:38:34.516 INFO:tasks.rados.rados.0.smithi138.stdout:206: expect (ObjNum 20 snap 0 seq_num 20) 2023-06-27T12:38:34.516 INFO:tasks.rados.rados.0.smithi138.stdout:207: snap_remove snap 12 2023-06-27T12:38:34.920 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:34.921 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:34.921 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:35.048 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:35.042+0000 7f3eb6163700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:35.051 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:35.048+0000 7f3eb6163700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:35.242 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:35.244 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:35.256 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2023-06-27T12:38:34.498979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":120103790,"num_objects":74,"num_object_clones":26,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":463,"num_read_kb":208942,"num_write":591,"num_write_kb":133380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":538,"ondisk_log_size":538,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":354212,"kb_used_data":246916,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377133148,"statfs":{"total":386547056640,"available":386184343552,"internally_reserved":0,"allocated":252841984,"data_stored":256189618,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10857190,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":12,"num_read_kb":13234,"num_write":8,"num_write_kb":2107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014889"},"pg_stats":[{"pgid":"3.f","version":"18'4","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.531587+0000","last_change":"2023-06-27T12:38:32.531996+0000","last_active":"2023-06-27T12:38:33.531587+0000","last_peered":"2023-06-27T12:38:33.531587+0000","last_clean":"2023-06-27T12:38:33.531587+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:33.531587+0000","last_undegraded":"2023-06-27T12:38:33.531587+0000","last_fullsized":"2023-06-27T12:38:33.531587+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"44'39","reported_seq":120,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.527150+0000","last_change":"2023-06-27T12:38:32.538373+0000","last_active":"2023-06-27T12:38:33.527150+0000","last_peered":"2023-06-27T12:38:33.527150+0000","last_clean":"2023-06-27T12:38:33.527150+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:33.527150+0000","last_undegraded":"2023-06-27T12:38:33.527150+0000","last_fullsized":"2023-06-27T12:38:33.527150+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9882417,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":23,"num_read_kb":17650,"num_write":36,"num_write_kb":10378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"38'31","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.525415+0000","last_change":"2023-06-27T12:38:33.536169+0000","last_active":"2023-06-27T12:38:33.525415+0000","last_peered":"2023-06-27T12:38:33.525415+0000","last_clean":"2023-06-27T12:38:33.525415+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:33.525415+0000","last_undegraded":"2023-06-27T12:38:33.525415+0000","last_fullsized":"2023-06-27T12:38:33.525415+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5831963,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":12452,"num_write":35,"num_write_kb":9601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"45'30","reported_seq":97,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.616067+0000","last_change":"2023-06-27T12:38:32.533023+0000","last_active":"2023-06-27T12:38:33.616067+0000","last_peered":"2023-06-27T12:38:33.616067+0000","last_clean":"2023-06-27T12:38:33.616067+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:33.616067+0000","last_undegraded":"2023-06-27T12:38:33.616067+0000","last_fullsized":"2023-06-27T12:38:33.616067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7613538,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":22,"num_read_kb":3229,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"39'13","reported_seq":89,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.791323+0000","last_change":"2023-06-27T12:38:32.583062+0000","last_active":"2023-06-27T12:38:33.791323+0000","last_peered":"2023-06-27T12:38:33.791323+0000","last_clean":"2023-06-27T12:38:33.791323+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:33.791323+0000","last_undegraded":"2023-06-27T12:38:33.791323+0000","last_fullsized":"2023-06-27T12:38:33.791323+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5831996,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":14489,"num_write":17,"num_write_kb":4485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"37'44","reported_seq":118,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.529817+0000","last_change":"2023-06-27T12:38:32.534151+0000","last_active":"2023-06-27T12:38:33.529817+0000","last_peered":"2023-06-27T12:38:33.529817+0000","last_clean":"2023-06-27T12:38:33.529817+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:33.529817+0000","last_undegraded":"2023-06-27T12:38:33.529817+0000","last_fullsized":"2023-06-27T12:38:33.529817+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327819,"num_objects":8,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":16659,"num_write":45,"num_write_kb":12861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"31'31","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.547791+0000","last_change":"2023-06-27T12:38:32.532356+0000","last_active":"2023-06-27T12:38:33.547791+0000","last_peered":"2023-06-27T12:38:33.547791+0000","last_clean":"2023-06-27T12:38:33.547791+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:33.547791+0000","last_undegraded":"2023-06-27T12:38:33.547791+0000","last_fullsized":"2023-06-27T12:38:33.547791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8739541,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17108,"num_write":30,"num_write_kb":8934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"28'8","reported_seq":81,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.530679+0000","last_change":"2023-06-27T12:38:33.535042+0000","last_active":"2023-06-27T12:38:33.530679+0000","last_peered":"2023-06-27T12:38:33.530679+0000","last_clean":"2023-06-27T12:38:33.530679+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:33.530679+0000","last_undegraded":"2023-06-27T12:38:33.530679+0000","last_fullsized":"2023-06-27T12:38:33.530679+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3327753,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":11710,"num_write":9,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.524117+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:33.524117+0000","last_peered":"2023-06-27T12:38:33.524117+0000","last_clean":"2023-06-27T12:38:33.524117+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:33.524117+0000","last_undegraded":"2023-06-27T12:38:33.524117+0000","last_fullsized":"2023-06-27T12:38:33.524117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.533082+0000","last_change":"2023-06-27T12:38:33.536610+0000","last_active":"2023-06-27T12:38:33.533082+0000","last_peered":"2023-06-27T12:38:33.533082+0000","last_clean":"2023-06-27T12:38:33.533082+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:33.533082+0000","last_undegraded":"2023-06-27T12:38:33.533082+0000","last_fullsized":"2023-06-27T12:38:33.533082+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":55,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.526795+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:33.526795+0000","last_peered":"2023-06-27T12:38:33.526795+0000","last_clean":"2023-06-27T12:38:33.526795+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:33.526795+0000","last_undegraded":"2023-06-27T12:38:33.526795+0000","last_fullsized":"2023-06-27T12:38:33.526795+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"43'29","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.536512+0000","last_change":"2023-06-27T12:38:32.624481+0000","last_active":"2023-06-27T12:38:33.536512+0000","last_peered":"2023-06-27T12:38:33.536512+0000","last_clean":"2023-06-27T12:38:33.536512+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:33.536512+0000","last_undegraded":"2023-06-27T12:38:33.536512+0000","last_fullsized":"2023-06-27T12:38:33.536512+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3109262,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":15,"num_read_kb":5448,"num_write":25,"num_write_kb":7014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.527631+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:33.527631+0000","last_peered":"2023-06-27T12:38:33.527631+0000","last_clean":"2023-06-27T12:38:33.527631+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:33.527631+0000","last_undegraded":"2023-06-27T12:38:33.527631+0000","last_fullsized":"2023-06-27T12:38:33.527631+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"39'55","reported_seq":146,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.791621+0000","last_change":"2023-06-27T12:38:32.546614+0000","last_active":"2023-06-27T12:38:33.791621+0000","last_peered":"2023-06-27T12:38:33.791621+0000","last_clean":"2023-06-27T12:38:33.791621+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:33.791621+0000","last_undegraded":"2023-06-27T12:38:33.791621+0000","last_fullsized":"2023-06-27T12:38:33.791621+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17227010,"num_objects":7,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":42,"num_read_kb":38611,"num_write":63,"num_write_kb":18831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.528326+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:33.528326+0000","last_peered":"2023-06-27T12:38:33.528326+0000","last_clean":"2023-06-27T12:38:33.528326+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:33.528326+0000","last_undegraded":"2023-06-27T12:38:33.528326+0000","last_fullsized":"2023-06-27T12:38:33.528326+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":188,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.796312+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:33.796312+0000","last_peered":"2023-06-27T12:38:33.796312+0000","last_clean":"2023-06-27T12:38:33.796312+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:33.796312+0000","last_undegraded":"2023-06-27T12:38:33.796312+0000","last_fullsized":"2023-06-27T12:38:33.796312+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"37'44","reported_seq":127,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.534031+0000","last_change":"2023-06-27T12:38:33.534031+0000","last_active":"2023-06-27T12:38:33.534031+0000","last_peered":"2023-06-27T12:38:33.534031+0000","last_clean":"2023-06-27T12:38:33.534031+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:33.534031+0000","last_undegraded":"2023-06-27T12:38:33.534031+0000","last_fullsized":"2023-06-27T12:38:33.534031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11932838,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":49,"num_read_kb":19114,"num_write":49,"num_write_kb":13644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.545388+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:33.545388+0000","last_peered":"2023-06-27T12:38:33.545388+0000","last_clean":"2023-06-27T12:38:33.545388+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:33.545388+0000","last_undegraded":"2023-06-27T12:38:33.545388+0000","last_fullsized":"2023-06-27T12:38:33.545388+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"39'29","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.788974+0000","last_change":"2023-06-27T12:38:32.549745+0000","last_active":"2023-06-27T12:38:33.788974+0000","last_peered":"2023-06-27T12:38:33.788974+0000","last_clean":"2023-06-27T12:38:33.788974+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:33.788974+0000","last_undegraded":"2023-06-27T12:38:33.788974+0000","last_fullsized":"2023-06-27T12:38:33.788974+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798382,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.794265+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:33.794265+0000","last_peered":"2023-06-27T12:38:33.794265+0000","last_clean":"2023-06-27T12:38:33.794265+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:33.794265+0000","last_undegraded":"2023-06-27T12:38:33.794265+0000","last_fullsized":"2023-06-27T12:38:33.794265+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"38'40","reported_seq":122,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.525052+0000","last_change":"2023-06-27T12:38:32.541769+0000","last_active":"2023-06-27T12:38:33.525052+0000","last_peered":"2023-06-27T12:38:33.525052+0000","last_clean":"2023-06-27T12:38:33.525052+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:33.525052+0000","last_undegraded":"2023-06-27T12:38:33.525052+0000","last_fullsized":"2023-06-27T12:38:33.525052+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13832029,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":15303,"num_write":41,"num_write_kb":12814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"33'10","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.800406+0000","last_change":"2023-06-27T12:38:32.547791+0000","last_active":"2023-06-27T12:38:33.800406+0000","last_peered":"2023-06-27T12:38:33.800406+0000","last_clean":"2023-06-27T12:38:33.800406+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:33.800406+0000","last_undegraded":"2023-06-27T12:38:33.800406+0000","last_fullsized":"2023-06-27T12:38:33.800406+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5915998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":3466,"num_write":13,"num_write_kb":3651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.527341+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:33.527341+0000","last_peered":"2023-06-27T12:38:33.527341+0000","last_clean":"2023-06-27T12:38:33.527341+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:33.527341+0000","last_undegraded":"2023-06-27T12:38:33.527341+0000","last_fullsized":"2023-06-27T12:38:33.527341+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.532757+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:33.532757+0000","last_peered":"2023-06-27T12:38:33.532757+0000","last_clean":"2023-06-27T12:38:33.532757+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:33.532757+0000","last_undegraded":"2023-06-27T12:38:33.532757+0000","last_fullsized":"2023-06-27T12:38:33.532757+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"37'34","reported_seq":112,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-06-27T12:38:33.526462+0000","last_change":"2023-06-27T12:38:32.537700+0000","last_active":"2023-06-27T12:38:33.526462+0000","last_peered":"2023-06-27T12:38:33.526462+0000","last_clean":"2023-06-27T12:38:33.526462+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:33.526462+0000","last_undegraded":"2023-06-27T12:38:33.526462+0000","last_fullsized":"2023-06-27T12:38:33.526462+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3142876,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":20,"num_read_kb":14986,"num_write":27,"num_write_kb":8065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":119513422,"num_objects":72,"num_object_clones":26,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":360,"num_read_kb":208851,"num_write":454,"num_write_kb":131486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":441,"ondisk_log_size":441,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91864,"kb_used_data":65040,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94279976,"statfs":{"total":96636764160,"available":96542695424,"internally_reserved":0,"allocated":66600960,"data_stored":66992810,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89404,"kb_used_data":62580,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94282436,"statfs":{"total":96636764160,"available":96545214464,"internally_reserved":0,"allocated":64081920,"data_stored":65358866,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":84004,"kb_used_data":57180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94287836,"statfs":{"total":96636764160,"available":96550744064,"internally_reserved":0,"allocated":58552320,"data_stored":59604579,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88940,"kb_used_data":62116,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94282900,"statfs":{"total":96636764160,"available":96545689600,"internally_reserved":0,"allocated":63606784,"data_stored":64233363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:35.504 INFO:tasks.rados.rados.0.smithi138.stdout:207: done (3 left) 2023-06-27T12:38:35.505 INFO:tasks.rados.rados.0.smithi138.stdout:208: read oid 20 snap -1 2023-06-27T12:38:35.505 INFO:tasks.rados.rados.0.smithi138.stdout:208: expect (ObjNum 114 snap 10 seq_num 114) 2023-06-27T12:38:35.505 INFO:tasks.rados.rados.0.smithi138.stdout:209: write (excl) oid 34 current snap is 15 2023-06-27T12:38:35.505 INFO:tasks.rados.rados.0.smithi138.stdout:209: seq_num 127 ranges {534489=618271,1565566=568923} 2023-06-27T12:38:35.511 INFO:tasks.rados.rados.0.smithi138.stdout:209: writing smithi138138488-34 from 534489 to 1152760 tid 1 2023-06-27T12:38:35.517 INFO:tasks.rados.rados.0.smithi138.stdout:209: writing smithi138138488-34 from 1565566 to 2134489 tid 2 2023-06-27T12:38:35.517 INFO:tasks.rados.rados.0.smithi138.stdout:210: read oid 22 snap -1 2023-06-27T12:38:35.517 INFO:tasks.rados.rados.0.smithi138.stdout:210: expect (ObjNum 68 snap 3 seq_num 68) 2023-06-27T12:38:35.518 INFO:tasks.rados.rados.0.smithi138.stdout:211: delete oid 48 current snap is 15 2023-06-27T12:38:35.518 INFO:tasks.rados.rados.0.smithi138.stdout:206: expect (ObjNum 20 snap 0 seq_num 20) 2023-06-27T12:38:35.612 INFO:tasks.rados.rados.0.smithi138.stdout:206: done (6 left) 2023-06-27T12:38:35.612 INFO:tasks.rados.rados.0.smithi138.stdout:211: done (5 left) 2023-06-27T12:38:35.612 INFO:tasks.rados.rados.0.smithi138.stdout:212: read oid 29 snap -1 2023-06-27T12:38:35.612 INFO:tasks.rados.rados.0.smithi138.stdout:212: expect (ObjNum 90 snap 7 seq_num 90) 2023-06-27T12:38:35.613 INFO:tasks.rados.rados.0.smithi138.stdout:213: write oid 37 current snap is 15 2023-06-27T12:38:35.613 INFO:tasks.rados.rados.0.smithi138.stdout:204: expect (ObjNum 15 snap 0 seq_num 15) 2023-06-27T12:38:35.707 INFO:tasks.rados.rados.0.smithi138.stdout:205: finishing write tid 1 to smithi138138488-7 2023-06-27T12:38:35.707 INFO:tasks.rados.rados.0.smithi138.stdout:205: finishing write tid 2 to smithi138138488-7 2023-06-27T12:38:35.707 INFO:tasks.rados.rados.0.smithi138.stdout:205: finishing write tid 3 to smithi138138488-7 2023-06-27T12:38:35.707 INFO:tasks.rados.rados.0.smithi138.stdout:205: finishing write tid 4 to smithi138138488-7 2023-06-27T12:38:35.707 INFO:tasks.rados.rados.0.smithi138.stdout:205: oid 7 updating version 0 to 35 2023-06-27T12:38:35.708 INFO:tasks.rados.rados.0.smithi138.stdout:205: oid 7 version 35 is already newer than 32 2023-06-27T12:38:35.708 INFO:tasks.rados.rados.0.smithi138.stdout:205: oid 7 version 35 is already newer than 35 2023-06-27T12:38:35.708 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 35 (ObjNum 126 snap 15 seq_num 126) dirty exists 2023-06-27T12:38:35.708 INFO:tasks.rados.rados.0.smithi138.stdout:205: left oid 7 (ObjNum 126 snap 15 seq_num 126) 2023-06-27T12:38:35.708 INFO:tasks.rados.rados.0.smithi138.stdout:213: seq_num 129 ranges {568103=768769,2082530=85573} 2023-06-27T12:38:35.714 INFO:tasks.rados.rados.0.smithi138.stdout:213: writing smithi138138488-37 from 568103 to 1336872 tid 1 2023-06-27T12:38:35.719 INFO:tasks.rados.rados.0.smithi138.stdout:213: writing smithi138138488-37 from 2082530 to 2168103 tid 2 2023-06-27T12:38:35.719 INFO:tasks.rados.rados.0.smithi138.stdout:204: done (6 left) 2023-06-27T12:38:35.719 INFO:tasks.rados.rados.0.smithi138.stdout:205: done (5 left) 2023-06-27T12:38:35.719 INFO:tasks.rados.rados.0.smithi138.stdout:214: write oid 23 current snap is 15 2023-06-27T12:38:35.719 INFO:tasks.rados.rados.0.smithi138.stdout:214: seq_num 130 ranges {584910=560371,1857365=327545} 2023-06-27T12:38:35.725 INFO:tasks.rados.rados.0.smithi138.stdout:214: writing smithi138138488-23 from 584910 to 1145281 tid 1 2023-06-27T12:38:35.731 INFO:tasks.rados.rados.0.smithi138.stdout:214: writing smithi138138488-23 from 1857365 to 2184910 tid 2 2023-06-27T12:38:35.732 INFO:tasks.rados.rados.0.smithi138.stdout:215: snap_remove snap 10 2023-06-27T12:38:36.103 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:36.230 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:36.225+0000 7f64cff84700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:36.233 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:36.229+0000 7f64cff84700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:36.511 INFO:tasks.rados.rados.0.smithi138.stdout:215: done (6 left) 2023-06-27T12:38:36.511 INFO:tasks.rados.rados.0.smithi138.stdout:216: delete oid 42 current snap is 15 2023-06-27T12:38:36.511 INFO:tasks.rados.rados.0.smithi138.stdout:210: expect (ObjNum 68 snap 3 seq_num 68) 2023-06-27T12:38:36.673 INFO:tasks.rados.rados.0.smithi138.stdout:208: expect (ObjNum 114 snap 10 seq_num 114) 2023-06-27T12:38:36.873 INFO:tasks.rados.rados.0.smithi138.stdout:209: finishing write tid 1 to smithi138138488-34 2023-06-27T12:38:36.873 INFO:tasks.rados.rados.0.smithi138.stdout:212: expect (ObjNum 90 snap 7 seq_num 90) 2023-06-27T12:38:37.049 INFO:tasks.rados.rados.0.smithi138.stdout:209: finishing write tid 2 to smithi138138488-34 2023-06-27T12:38:37.049 INFO:tasks.rados.rados.0.smithi138.stdout:209: finishing write tid 3 to smithi138138488-34 2023-06-27T12:38:37.050 INFO:tasks.rados.rados.0.smithi138.stdout:209: finishing write tid 4 to smithi138138488-34 2023-06-27T12:38:37.050 INFO:tasks.rados.rados.0.smithi138.stdout:209: oid 34 updating version 0 to 48 2023-06-27T12:38:37.050 INFO:tasks.rados.rados.0.smithi138.stdout:209: oid 34 version 48 is already newer than 48 2023-06-27T12:38:37.050 INFO:tasks.rados.rados.0.smithi138.stdout:209: oid 34 version 48 is already newer than 45 2023-06-27T12:38:37.050 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 48 (ObjNum 127 snap 15 seq_num 127) dirty exists 2023-06-27T12:38:37.050 INFO:tasks.rados.rados.0.smithi138.stdout:209: left oid 34 (ObjNum 127 snap 15 seq_num 127) 2023-06-27T12:38:37.050 INFO:tasks.rados.rados.0.smithi138.stdout:208: done (6 left) 2023-06-27T12:38:37.051 INFO:tasks.rados.rados.0.smithi138.stdout:209: done (5 left) 2023-06-27T12:38:37.051 INFO:tasks.rados.rados.0.smithi138.stdout:210: done (4 left) 2023-06-27T12:38:37.051 INFO:tasks.rados.rados.0.smithi138.stdout:212: done (3 left) 2023-06-27T12:38:37.051 INFO:tasks.rados.rados.0.smithi138.stdout:216: done (2 left) 2023-06-27T12:38:37.051 INFO:tasks.rados.rados.0.smithi138.stdout:217: write (excl) oid 20 current snap is 15 2023-06-27T12:38:37.051 INFO:tasks.rados.rados.0.smithi138.stdout:213: finishing write tid 1 to smithi138138488-37 2023-06-27T12:38:37.051 INFO:tasks.rados.rados.0.smithi138.stdout:213: finishing write tid 2 to smithi138138488-37 2023-06-27T12:38:37.051 INFO:tasks.rados.rados.0.smithi138.stdout:214: finishing write tid 1 to smithi138138488-23 2023-06-27T12:38:37.052 INFO:tasks.rados.rados.0.smithi138.stdout:213: finishing write tid 3 to smithi138138488-37 2023-06-27T12:38:37.052 INFO:tasks.rados.rados.0.smithi138.stdout:213: finishing write tid 4 to smithi138138488-37 2023-06-27T12:38:37.052 INFO:tasks.rados.rados.0.smithi138.stdout:213: oid 37 updating version 0 to 35 2023-06-27T12:38:37.052 INFO:tasks.rados.rados.0.smithi138.stdout:213: oid 37 version 35 is already newer than 35 2023-06-27T12:38:37.052 INFO:tasks.rados.rados.0.smithi138.stdout:213: oid 37 version 35 is already newer than 35 2023-06-27T12:38:37.052 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 35 (ObjNum 129 snap 15 seq_num 129) dirty exists 2023-06-27T12:38:37.052 INFO:tasks.rados.rados.0.smithi138.stdout:213: left oid 37 (ObjNum 129 snap 15 seq_num 129) 2023-06-27T12:38:37.053 INFO:tasks.rados.rados.0.smithi138.stdout:214: finishing write tid 2 to smithi138138488-23 2023-06-27T12:38:37.053 INFO:tasks.rados.rados.0.smithi138.stdout:214: finishing write tid 3 to smithi138138488-23 2023-06-27T12:38:37.053 INFO:tasks.rados.rados.0.smithi138.stdout:214: finishing write tid 4 to smithi138138488-23 2023-06-27T12:38:37.053 INFO:tasks.rados.rados.0.smithi138.stdout:214: oid 23 updating version 0 to 34 2023-06-27T12:38:37.053 INFO:tasks.rados.rados.0.smithi138.stdout:214: oid 23 version 34 is already newer than 34 2023-06-27T12:38:37.053 INFO:tasks.rados.rados.0.smithi138.stdout:214: oid 23 version 34 is already newer than 34 2023-06-27T12:38:37.053 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 34 (ObjNum 130 snap 15 seq_num 130) dirty exists 2023-06-27T12:38:37.053 INFO:tasks.rados.rados.0.smithi138.stdout:214: left oid 23 (ObjNum 130 snap 15 seq_num 130) 2023-06-27T12:38:37.054 INFO:tasks.rados.rados.0.smithi138.stdout:217: seq_num 132 ranges {618524=710869,1857976=360548} 2023-06-27T12:38:37.056 INFO:tasks.rados.rados.0.smithi138.stdout:217: writing smithi138138488-20 from 618524 to 1329393 tid 1 2023-06-27T12:38:37.061 INFO:tasks.rados.rados.0.smithi138.stdout:217: writing smithi138138488-20 from 1857976 to 2218524 tid 2 2023-06-27T12:38:37.061 INFO:tasks.rados.rados.0.smithi138.stdout:213: done (2 left) 2023-06-27T12:38:37.061 INFO:tasks.rados.rados.0.smithi138.stdout:214: done (1 left) 2023-06-27T12:38:37.061 INFO:tasks.rados.rados.0.smithi138.stdout:218: delete oid 38 current snap is 15 2023-06-27T12:38:37.106 INFO:tasks.rados.rados.0.smithi138.stdout:218: done (1 left) 2023-06-27T12:38:37.106 INFO:tasks.rados.rados.0.smithi138.stdout:219: read oid 16 snap -1 2023-06-27T12:38:37.106 INFO:tasks.rados.rados.0.smithi138.stdout:219: expect (ObjNum 15 snap 0 seq_num 15) 2023-06-27T12:38:37.106 INFO:tasks.rados.rados.0.smithi138.stdout:220: write (excl) oid 15 current snap is 15 2023-06-27T12:38:37.106 INFO:tasks.rados.rados.0.smithi138.stdout:220: seq_num 134 ranges {652138=461367,1574940=677198} 2023-06-27T12:38:37.112 INFO:tasks.rados.rados.0.smithi138.stdout:220: writing smithi138138488-15 from 652138 to 1113505 tid 1 2023-06-27T12:38:37.119 INFO:tasks.rados.rados.0.smithi138.stdout:220: writing smithi138138488-15 from 1574940 to 2252138 tid 2 2023-06-27T12:38:37.119 INFO:tasks.rados.rados.0.smithi138.stdout:221: snap_create 2023-06-27T12:38:37.120 INFO:tasks.rados.rados.0.smithi138.stdout:217: finishing write tid 1 to smithi138138488-20 2023-06-27T12:38:37.120 INFO:tasks.rados.rados.0.smithi138.stdout:219: expect (ObjNum 15 snap 0 seq_num 15) 2023-06-27T12:38:37.220 INFO:tasks.rados.rados.0.smithi138.stdout:217: finishing write tid 2 to smithi138138488-20 2023-06-27T12:38:37.220 INFO:tasks.rados.rados.0.smithi138.stdout:220: finishing write tid 1 to smithi138138488-15 2023-06-27T12:38:37.220 INFO:tasks.rados.rados.0.smithi138.stdout:217: finishing write tid 3 to smithi138138488-20 2023-06-27T12:38:37.221 INFO:tasks.rados.rados.0.smithi138.stdout:217: finishing write tid 4 to smithi138138488-20 2023-06-27T12:38:37.221 INFO:tasks.rados.rados.0.smithi138.stdout:217: oid 20 updating version 0 to 40 2023-06-27T12:38:37.221 INFO:tasks.rados.rados.0.smithi138.stdout:217: oid 20 updating version 40 to 43 2023-06-27T12:38:37.221 INFO:tasks.rados.rados.0.smithi138.stdout:217: oid 20 version 43 is already newer than 43 2023-06-27T12:38:37.221 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 43 (ObjNum 132 snap 15 seq_num 132) dirty exists 2023-06-27T12:38:37.221 INFO:tasks.rados.rados.0.smithi138.stdout:217: left oid 20 (ObjNum 132 snap 15 seq_num 132) 2023-06-27T12:38:37.221 INFO:tasks.rados.rados.0.smithi138.stdout:220: finishing write tid 2 to smithi138138488-15 2023-06-27T12:38:37.225 INFO:tasks.rados.rados.0.smithi138.stdout:220: finishing write tid 3 to smithi138138488-15 2023-06-27T12:38:37.226 INFO:tasks.rados.rados.0.smithi138.stdout:220: finishing write tid 4 to smithi138138488-15 2023-06-27T12:38:37.226 INFO:tasks.rados.rados.0.smithi138.stdout:220: oid 15 updating version 0 to 17 2023-06-27T12:38:37.226 INFO:tasks.rados.rados.0.smithi138.stdout:220: oid 15 version 17 is already newer than 14 2023-06-27T12:38:37.227 INFO:tasks.rados.rados.0.smithi138.stdout:220: oid 15 version 17 is already newer than 17 2023-06-27T12:38:37.227 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 17 (ObjNum 134 snap 15 seq_num 134) dirty exists 2023-06-27T12:38:37.227 INFO:tasks.rados.rados.0.smithi138.stdout:220: left oid 15 (ObjNum 134 snap 15 seq_num 134) 2023-06-27T12:38:37.514 INFO:tasks.rados.rados.0.smithi138.stdout:217: done (3 left) 2023-06-27T12:38:37.514 INFO:tasks.rados.rados.0.smithi138.stdout:219: done (2 left) 2023-06-27T12:38:37.515 INFO:tasks.rados.rados.0.smithi138.stdout:220: done (1 left) 2023-06-27T12:38:37.515 INFO:tasks.rados.rados.0.smithi138.stdout:221: done (0 left) 2023-06-27T12:38:37.515 INFO:tasks.rados.rados.0.smithi138.stdout:222: snap_create 2023-06-27T12:38:38.303 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:38.431 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:38.426+0000 7fce02f19700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:38.434 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:38.431+0000 7fce02f19700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:38.519 INFO:tasks.rados.rados.0.smithi138.stdout:222: done (0 left) 2023-06-27T12:38:38.520 INFO:tasks.rados.rados.0.smithi138.stdout:223: write (excl) oid 45 current snap is 17 2023-06-27T12:38:38.520 INFO:tasks.rados.rados.0.smithi138.stdout:223: seq_num 135 ranges {668945=536616,1633422=491363} 2023-06-27T12:38:38.527 INFO:tasks.rados.rados.0.smithi138.stdout:223: writing smithi138138488-45 from 668945 to 1205561 tid 1 2023-06-27T12:38:38.535 INFO:tasks.rados.rados.0.smithi138.stdout:223: writing smithi138138488-45 from 1633422 to 2124785 tid 2 2023-06-27T12:38:38.535 INFO:tasks.rados.rados.0.smithi138.stdout:224: write oid 25 current snap is 17 2023-06-27T12:38:38.535 INFO:tasks.rados.rados.0.smithi138.stdout:224: seq_num 136 ranges {685752=611865,1975551=310201} 2023-06-27T12:38:38.542 INFO:tasks.rados.rados.0.smithi138.stdout:224: writing smithi138138488-25 from 685752 to 1297617 tid 1 2023-06-27T12:38:38.547 INFO:tasks.rados.rados.0.smithi138.stdout:224: writing smithi138138488-25 from 1975551 to 2285752 tid 2 2023-06-27T12:38:38.547 INFO:tasks.rados.rados.0.smithi138.stdout:225: write (excl) oid 19 current snap is 17 2023-06-27T12:38:38.547 INFO:tasks.rados.rados.0.smithi138.stdout:225: seq_num 137 ranges {702559=403467,1750386=495032} 2023-06-27T12:38:38.553 INFO:tasks.rados.rados.0.smithi138.stdout:225: writing smithi138138488-19 from 702559 to 1106026 tid 1 2023-06-27T12:38:38.560 INFO:tasks.rados.rados.0.smithi138.stdout:225: writing smithi138138488-19 from 1750386 to 2245418 tid 2 2023-06-27T12:38:38.561 INFO:tasks.rados.rados.0.smithi138.stdout:226: read oid 44 snap -1 2023-06-27T12:38:38.561 INFO:tasks.rados.rados.0.smithi138.stdout:226: expect (ObjNum 105 snap 9 seq_num 105) 2023-06-27T12:38:38.561 INFO:tasks.rados.rados.0.smithi138.stdout:227: read oid 32 snap -1 2023-06-27T12:38:38.561 INFO:tasks.rados.rados.0.smithi138.stdout:227: expect (ObjNum 31 snap 0 seq_num 31) 2023-06-27T12:38:38.561 INFO:tasks.rados.rados.0.smithi138.stdout:228: read oid 36 snap -1 2023-06-27T12:38:38.561 INFO:tasks.rados.rados.0.smithi138.stdout:228: expect deleted 2023-06-27T12:38:38.561 INFO:tasks.rados.rados.0.smithi138.stdout:229: read oid 6 snap -1 2023-06-27T12:38:38.562 INFO:tasks.rados.rados.0.smithi138.stdout:229: expect (ObjNum 66 snap 3 seq_num 66) 2023-06-27T12:38:38.562 INFO:tasks.rados.rados.0.smithi138.stdout:230: snap_remove snap 15 2023-06-27T12:38:39.521 INFO:tasks.rados.rados.0.smithi138.stdout:230: done (7 left) 2023-06-27T12:38:39.521 INFO:tasks.rados.rados.0.smithi138.stdout:231: read oid 31 snap -1 2023-06-27T12:38:39.521 INFO:tasks.rados.rados.0.smithi138.stdout:231: expect (ObjNum 30 snap 0 seq_num 30) 2023-06-27T12:38:39.521 INFO:tasks.rados.rados.0.smithi138.stdout:232: write oid 2 current snap is 17 2023-06-27T12:38:39.522 INFO:tasks.rados.rados.0.smithi138.stdout:232: seq_num 138 ranges {719366=478716,1808868=510498} 2023-06-27T12:38:39.528 INFO:tasks.rados.rados.0.smithi138.stdout:232: writing smithi138138488-2 from 719366 to 1198082 tid 1 2023-06-27T12:38:39.536 INFO:tasks.rados.rados.0.smithi138.stdout:232: writing smithi138138488-2 from 1808868 to 2319366 tid 2 2023-06-27T12:38:39.536 INFO:tasks.rados.rados.0.smithi138.stdout:233: read oid 29 snap -1 2023-06-27T12:38:39.537 INFO:tasks.rados.rados.0.smithi138.stdout:233: expect (ObjNum 90 snap 7 seq_num 90) 2023-06-27T12:38:39.537 INFO:tasks.rados.rados.0.smithi138.stdout:234: write oid 14 current snap is 17 2023-06-27T12:38:39.537 INFO:tasks.rados.rados.0.smithi138.stdout:234: seq_num 139 ranges {736173=553965,1867350=468823} 2023-06-27T12:38:39.545 INFO:tasks.rados.rados.0.smithi138.stdout:234: writing smithi138138488-14 from 736173 to 1290138 tid 1 2023-06-27T12:38:39.553 INFO:tasks.rados.rados.0.smithi138.stdout:234: writing smithi138138488-14 from 1867350 to 2336173 tid 2 2023-06-27T12:38:39.553 INFO:tasks.rados.rados.0.smithi138.stdout:235: delete oid 16 current snap is 17 2023-06-27T12:38:39.553 INFO:tasks.rados.rados.0.smithi138.stdout:226: expect (ObjNum 105 snap 9 seq_num 105) 2023-06-27T12:38:39.793 INFO:tasks.rados.rados.0.smithi138.stdout:223: finishing write tid 1 to smithi138138488-45 2023-06-27T12:38:39.793 INFO:tasks.rados.rados.0.smithi138.stdout:226: done (11 left) 2023-06-27T12:38:39.793 INFO:tasks.rados.rados.0.smithi138.stdout:235: done (10 left) 2023-06-27T12:38:39.793 INFO:tasks.rados.rados.0.smithi138.stdout:236: read oid 17 snap -1 2023-06-27T12:38:39.793 INFO:tasks.rados.rados.0.smithi138.stdout:236: expect deleted 2023-06-27T12:38:39.793 INFO:tasks.rados.rados.0.smithi138.stdout:237: delete oid 11 current snap is 17 2023-06-27T12:38:39.794 INFO:tasks.rados.rados.0.smithi138.stdout:229: expect (ObjNum 66 snap 3 seq_num 66) 2023-06-27T12:38:39.949 INFO:tasks.rados.rados.0.smithi138.stdout:225: finishing write tid 1 to smithi138138488-19 2023-06-27T12:38:39.950 INFO:tasks.rados.rados.0.smithi138.stdout:224: finishing write tid 1 to smithi138138488-25 2023-06-27T12:38:39.950 INFO:tasks.rados.rados.0.smithi138.stdout:224: finishing write tid 2 to smithi138138488-25 2023-06-27T12:38:39.950 INFO:tasks.rados.rados.0.smithi138.stdout:224: finishing write tid 3 to smithi138138488-25 2023-06-27T12:38:39.950 INFO:tasks.rados.rados.0.smithi138.stdout:224: finishing write tid 4 to smithi138138488-25 2023-06-27T12:38:39.950 INFO:tasks.rados.rados.0.smithi138.stdout:224: oid 25 updating version 0 to 14 2023-06-27T12:38:39.951 INFO:tasks.rados.rados.0.smithi138.stdout:224: oid 25 version 14 is already newer than 14 2023-06-27T12:38:39.951 INFO:tasks.rados.rados.0.smithi138.stdout:224: oid 25 version 14 is already newer than 14 2023-06-27T12:38:39.951 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 14 (ObjNum 136 snap 17 seq_num 136) dirty exists 2023-06-27T12:38:39.951 INFO:tasks.rados.rados.0.smithi138.stdout:224: left oid 25 (ObjNum 136 snap 17 seq_num 136) 2023-06-27T12:38:39.951 INFO:tasks.rados.rados.0.smithi138.stdout:223: finishing write tid 2 to smithi138138488-45 2023-06-27T12:38:39.951 INFO:tasks.rados.rados.0.smithi138.stdout:225: finishing write tid 2 to smithi138138488-19 2023-06-27T12:38:39.951 INFO:tasks.rados.rados.0.smithi138.stdout:225: finishing write tid 3 to smithi138138488-19 2023-06-27T12:38:39.951 INFO:tasks.rados.rados.0.smithi138.stdout:225: finishing write tid 4 to smithi138138488-19 2023-06-27T12:38:39.952 INFO:tasks.rados.rados.0.smithi138.stdout:225: oid 19 updating version 0 to 49 2023-06-27T12:38:39.952 INFO:tasks.rados.rados.0.smithi138.stdout:225: oid 19 updating version 49 to 51 2023-06-27T12:38:39.952 INFO:tasks.rados.rados.0.smithi138.stdout:225: oid 19 version 51 is already newer than 51 2023-06-27T12:38:39.952 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 51 (ObjNum 137 snap 17 seq_num 137) dirty exists 2023-06-27T12:38:39.952 INFO:tasks.rados.rados.0.smithi138.stdout:225: left oid 19 (ObjNum 137 snap 17 seq_num 137) 2023-06-27T12:38:39.952 INFO:tasks.rados.rados.0.smithi138.stdout:223: finishing write tid 3 to smithi138138488-45 2023-06-27T12:38:39.952 INFO:tasks.rados.rados.0.smithi138.stdout:224: done (11 left) 2023-06-27T12:38:39.953 INFO:tasks.rados.rados.0.smithi138.stdout:225: done (10 left) 2023-06-27T12:38:39.953 INFO:tasks.rados.rados.0.smithi138.stdout:228: done (9 left) 2023-06-27T12:38:39.953 INFO:tasks.rados.rados.0.smithi138.stdout:229: done (8 left) 2023-06-27T12:38:39.953 INFO:tasks.rados.rados.0.smithi138.stdout:237: done (7 left) 2023-06-27T12:38:39.953 INFO:tasks.rados.rados.0.smithi138.stdout:238: snap_remove snap 11 2023-06-27T12:38:39.953 INFO:tasks.rados.rados.0.smithi138.stdout:223: finishing write tid 4 to smithi138138488-45 2023-06-27T12:38:39.953 INFO:tasks.rados.rados.0.smithi138.stdout:223: oid 45 updating version 0 to 59 2023-06-27T12:38:39.953 INFO:tasks.rados.rados.0.smithi138.stdout:223: oid 45 version 59 is already newer than 59 2023-06-27T12:38:39.954 INFO:tasks.rados.rados.0.smithi138.stdout:223: oid 45 version 59 is already newer than 56 2023-06-27T12:38:39.954 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 59 (ObjNum 135 snap 17 seq_num 135) dirty exists 2023-06-27T12:38:39.954 INFO:tasks.rados.rados.0.smithi138.stdout:223: left oid 45 (ObjNum 135 snap 17 seq_num 135) 2023-06-27T12:38:39.954 INFO:tasks.rados.rados.0.smithi138.stdout:227: expect (ObjNum 31 snap 0 seq_num 31) 2023-06-27T12:38:40.042 INFO:tasks.rados.rados.0.smithi138.stdout:231: expect (ObjNum 30 snap 0 seq_num 30) 2023-06-27T12:38:40.134 INFO:tasks.rados.rados.0.smithi138.stdout:233: expect (ObjNum 90 snap 7 seq_num 90) 2023-06-27T12:38:40.258 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:40.259 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:40.259 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:40.314 INFO:tasks.rados.rados.0.smithi138.stdout:232: finishing write tid 1 to smithi138138488-2 2023-06-27T12:38:40.314 INFO:tasks.rados.rados.0.smithi138.stdout:234: finishing write tid 1 to smithi138138488-14 2023-06-27T12:38:40.387 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:40.380+0000 7faebdb79700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:40.390 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:40.386+0000 7faebdb79700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:40.505 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:40.528 INFO:tasks.rados.rados.0.smithi138.stdout:223: done (7 left) 2023-06-27T12:38:40.528 INFO:tasks.rados.rados.0.smithi138.stdout:227: done (6 left) 2023-06-27T12:38:40.528 INFO:tasks.rados.rados.0.smithi138.stdout:231: done (5 left) 2023-06-27T12:38:40.528 INFO:tasks.rados.rados.0.smithi138.stdout:233: done (4 left) 2023-06-27T12:38:40.529 INFO:tasks.rados.rados.0.smithi138.stdout:238: done (3 left) 2023-06-27T12:38:40.529 INFO:tasks.rados.rados.0.smithi138.stdout:239: read oid 11 snap -1 2023-06-27T12:38:40.529 INFO:tasks.rados.rados.0.smithi138.stdout:239: expect deleted 2023-06-27T12:38:40.529 INFO:tasks.rados.rados.0.smithi138.stdout:240: snap_remove snap 16 2023-06-27T12:38:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:40.595 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:40.611 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2023-06-27T12:38:40.523308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112860237,"num_objects":83,"num_object_clones":34,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":517,"num_read_kb":254162,"num_write":627,"num_write_kb":142203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":585,"ondisk_log_size":585,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":380944,"kb_used_data":273648,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377106416,"statfs":{"total":386547056640,"available":386156969984,"internally_reserved":0,"allocated":280215552,"data_stored":291815544,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7243553,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":45220,"num_write":36,"num_write_kb":8823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016924"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":80,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.569275+0000","last_change":"2023-06-27T12:38:36.524273+0000","last_active":"2023-06-27T12:38:38.569275+0000","last_peered":"2023-06-27T12:38:38.569275+0000","last_clean":"2023-06-27T12:38:38.569275+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:38.569275+0000","last_undegraded":"2023-06-27T12:38:38.569275+0000","last_fullsized":"2023-06-27T12:38:38.569275+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"48'43","reported_seq":141,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.579304+0000","last_change":"2023-06-27T12:38:36.561883+0000","last_active":"2023-06-27T12:38:38.579304+0000","last_peered":"2023-06-27T12:38:38.579304+0000","last_clean":"2023-06-27T12:38:38.579304+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:38.579304+0000","last_undegraded":"2023-06-27T12:38:38.579304+0000","last_fullsized":"2023-06-27T12:38:38.579304+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8184943,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":27,"num_read_kb":20813,"num_write":40,"num_write_kb":11426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"47'35","reported_seq":126,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.525882+0000","last_change":"2023-06-27T12:38:36.572892+0000","last_active":"2023-06-27T12:38:38.525882+0000","last_peered":"2023-06-27T12:38:38.525882+0000","last_clean":"2023-06-27T12:38:38.525882+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:38.525882+0000","last_undegraded":"2023-06-27T12:38:38.525882+0000","last_fullsized":"2023-06-27T12:38:38.525882+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5092488,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":12453,"num_write":39,"num_write_kb":10436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":113,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.579993+0000","last_change":"2023-06-27T12:38:36.550023+0000","last_active":"2023-06-27T12:38:38.579993+0000","last_peered":"2023-06-27T12:38:38.579993+0000","last_clean":"2023-06-27T12:38:38.579993+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:38.579993+0000","last_undegraded":"2023-06-27T12:38:38.579993+0000","last_fullsized":"2023-06-27T12:38:38.579993+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":23,"num_read_kb":3229,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"48'17","reported_seq":106,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.544268+0000","last_change":"2023-06-27T12:38:36.569662+0000","last_active":"2023-06-27T12:38:38.544268+0000","last_peered":"2023-06-27T12:38:38.544268+0000","last_clean":"2023-06-27T12:38:38.544268+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:38.544268+0000","last_undegraded":"2023-06-27T12:38:38.544268+0000","last_fullsized":"2023-06-27T12:38:38.544268+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268978,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":14490,"num_write":21,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"47'48","reported_seq":145,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.595805+0000","last_change":"2023-06-27T12:38:36.525103+0000","last_active":"2023-06-27T12:38:38.595805+0000","last_peered":"2023-06-27T12:38:38.595805+0000","last_clean":"2023-06-27T12:38:38.595805+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:38.595805+0000","last_undegraded":"2023-06-27T12:38:38.595805+0000","last_fullsized":"2023-06-27T12:38:38.595805+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13882450,"num_objects":9,"num_object_clones":4,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":34372,"num_write":49,"num_write_kb":14021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"46'35","reported_seq":121,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.578984+0000","last_change":"2023-06-27T12:38:36.559796+0000","last_active":"2023-06-27T12:38:38.578984+0000","last_peered":"2023-06-27T12:38:38.578984+0000","last_clean":"2023-06-27T12:38:38.578984+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:38.578984+0000","last_undegraded":"2023-06-27T12:38:38.578984+0000","last_fullsized":"2023-06-27T12:38:38.578984+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731154,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":17109,"num_write":34,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"28'8","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.531194+0000","last_change":"2023-06-27T12:38:36.556200+0000","last_active":"2023-06-27T12:38:38.531194+0000","last_peered":"2023-06-27T12:38:38.531194+0000","last_clean":"2023-06-27T12:38:38.531194+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:38.531194+0000","last_undegraded":"2023-06-27T12:38:38.531194+0000","last_fullsized":"2023-06-27T12:38:38.531194+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3327753,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":11710,"num_write":9,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":77,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.586106+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:38.586106+0000","last_peered":"2023-06-27T12:38:38.586106+0000","last_clean":"2023-06-27T12:38:38.586106+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:38.586106+0000","last_undegraded":"2023-06-27T12:38:38.586106+0000","last_fullsized":"2023-06-27T12:38:38.586106+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":77,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.524493+0000","last_change":"2023-06-27T12:38:36.566231+0000","last_active":"2023-06-27T12:38:38.524493+0000","last_peered":"2023-06-27T12:38:38.524493+0000","last_clean":"2023-06-27T12:38:38.524493+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:38.524493+0000","last_undegraded":"2023-06-27T12:38:38.524493+0000","last_fullsized":"2023-06-27T12:38:38.524493+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.570196+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:38.570196+0000","last_peered":"2023-06-27T12:38:38.570196+0000","last_clean":"2023-06-27T12:38:38.570196+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:38.570196+0000","last_undegraded":"2023-06-27T12:38:38.570196+0000","last_fullsized":"2023-06-27T12:38:38.570196+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"47'34","reported_seq":125,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.618645+0000","last_change":"2023-06-27T12:38:36.560986+0000","last_active":"2023-06-27T12:38:38.618645+0000","last_peered":"2023-06-27T12:38:38.618645+0000","last_clean":"2023-06-27T12:38:38.618645+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:38.618645+0000","last_undegraded":"2023-06-27T12:38:38.618645+0000","last_fullsized":"2023-06-27T12:38:38.618645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294172,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":18,"num_read_kb":10007,"num_write":29,"num_write_kb":7882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.522060+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:38.522060+0000","last_peered":"2023-06-27T12:38:38.522060+0000","last_clean":"2023-06-27T12:38:38.522060+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:38.522060+0000","last_undegraded":"2023-06-27T12:38:38.522060+0000","last_fullsized":"2023-06-27T12:38:38.522060+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"50'59","reported_seq":165,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.698432+0000","last_change":"2023-06-27T12:38:36.548887+0000","last_active":"2023-06-27T12:38:38.698432+0000","last_peered":"2023-06-27T12:38:38.698432+0000","last_clean":"2023-06-27T12:38:38.698432+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:38.698432+0000","last_undegraded":"2023-06-27T12:38:38.698432+0000","last_fullsized":"2023-06-27T12:38:38.698432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15546343,"num_objects":8,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":43536,"num_write":67,"num_write_kb":19836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.524929+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:38.524929+0000","last_peered":"2023-06-27T12:38:38.524929+0000","last_clean":"2023-06-27T12:38:38.524929+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:38.524929+0000","last_undegraded":"2023-06-27T12:38:38.524929+0000","last_fullsized":"2023-06-27T12:38:38.524929+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":198,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.541881+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:38.541881+0000","last_peered":"2023-06-27T12:38:38.541881+0000","last_clean":"2023-06-27T12:38:38.541881+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:38.541881+0000","last_undegraded":"2023-06-27T12:38:38.541881+0000","last_fullsized":"2023-06-27T12:38:38.541881+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'51","reported_seq":149,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.637732+0000","last_change":"2023-06-27T12:38:36.555376+0000","last_active":"2023-06-27T12:38:38.637732+0000","last_peered":"2023-06-27T12:38:38.637732+0000","last_clean":"2023-06-27T12:38:38.637732+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:38.637732+0000","last_undegraded":"2023-06-27T12:38:38.637732+0000","last_fullsized":"2023-06-27T12:38:38.637732+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14235397,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":58,"num_read_kb":24513,"num_write":53,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.575916+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:38.575916+0000","last_peered":"2023-06-27T12:38:38.575916+0000","last_clean":"2023-06-27T12:38:38.575916+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:38.575916+0000","last_undegraded":"2023-06-27T12:38:38.575916+0000","last_fullsized":"2023-06-27T12:38:38.575916+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"39'29","reported_seq":119,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.544895+0000","last_change":"2023-06-27T12:38:36.551146+0000","last_active":"2023-06-27T12:38:38.544895+0000","last_peered":"2023-06-27T12:38:38.544895+0000","last_clean":"2023-06-27T12:38:38.544895+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:38.544895+0000","last_undegraded":"2023-06-27T12:38:38.544895+0000","last_fullsized":"2023-06-27T12:38:38.544895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798382,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.542536+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:38.542536+0000","last_peered":"2023-06-27T12:38:38.542536+0000","last_clean":"2023-06-27T12:38:38.542536+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:38.542536+0000","last_undegraded":"2023-06-27T12:38:38.542536+0000","last_fullsized":"2023-06-27T12:38:38.542536+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"38'40","reported_seq":135,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.589226+0000","last_change":"2023-06-27T12:38:36.528244+0000","last_active":"2023-06-27T12:38:38.589226+0000","last_peered":"2023-06-27T12:38:38.589226+0000","last_clean":"2023-06-27T12:38:38.589226+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:38.589226+0000","last_undegraded":"2023-06-27T12:38:38.589226+0000","last_fullsized":"2023-06-27T12:38:38.589226+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13832029,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":15303,"num_write":41,"num_write_kb":12814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"50'14","reported_seq":98,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.580951+0000","last_change":"2023-06-27T12:38:36.549651+0000","last_active":"2023-06-27T12:38:38.580951+0000","last_peered":"2023-06-27T12:38:38.580951+0000","last_clean":"2023-06-27T12:38:38.580951+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:38.580951+0000","last_undegraded":"2023-06-27T12:38:38.580951+0000","last_fullsized":"2023-06-27T12:38:38.580951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5798382,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8596,"num_write":17,"num_write_kb":4552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":77,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.582277+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:38.582277+0000","last_peered":"2023-06-27T12:38:38.582277+0000","last_clean":"2023-06-27T12:38:38.582277+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:38.582277+0000","last_undegraded":"2023-06-27T12:38:38.582277+0000","last_fullsized":"2023-06-27T12:38:38.582277+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.532207+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:38.532207+0000","last_peered":"2023-06-27T12:38:38.532207+0000","last_clean":"2023-06-27T12:38:38.532207+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:38.532207+0000","last_undegraded":"2023-06-27T12:38:38.532207+0000","last_fullsized":"2023-06-27T12:38:38.532207+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"47'35","reported_seq":127,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-06-27T12:38:38.578219+0000","last_change":"2023-06-27T12:38:36.527219+0000","last_active":"2023-06-27T12:38:38.578219+0000","last_peered":"2023-06-27T12:38:38.578219+0000","last_clean":"2023-06-27T12:38:38.578219+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:38.578219+0000","last_undegraded":"2023-06-27T12:38:38.578219+0000","last_fullsized":"2023-06-27T12:38:38.578219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3142876,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":22,"num_read_kb":19314,"num_write":27,"num_write_kb":8065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":112269869,"num_objects":81,"num_object_clones":34,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":414,"num_read_kb":254071,"num_write":490,"num_write_kb":140309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":488,"ondisk_log_size":488,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98580,"kb_used_data":71756,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94273260,"statfs":{"total":96636764160,"available":96535818240,"internally_reserved":0,"allocated":73478144,"data_stored":76076869,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95668,"kb_used_data":68844,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94276172,"statfs":{"total":96636764160,"available":96538800128,"internally_reserved":0,"allocated":70496256,"data_stored":73108522,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705710,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90844,"kb_used_data":64020,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94280996,"statfs":{"total":96636764160,"available":96543739904,"internally_reserved":0,"allocated":65556480,"data_stored":68881842,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":95852,"kb_used_data":69028,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94275988,"statfs":{"total":96636764160,"available":96538611712,"internally_reserved":0,"allocated":70684672,"data_stored":73748311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:40.642 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:40.637+0000 7fdbf3cc0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:40.645 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:40.642+0000 7fdbf3cc0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:41.530 INFO:tasks.rados.rados.0.smithi138.stdout:240: done (4 left) 2023-06-27T12:38:41.530 INFO:tasks.rados.rados.0.smithi138.stdout:241: read oid 16 snap -1 2023-06-27T12:38:41.530 INFO:tasks.rados.rados.0.smithi138.stdout:241: expect deleted 2023-06-27T12:38:41.530 INFO:tasks.rados.rados.0.smithi138.stdout:242: read oid 21 snap -1 2023-06-27T12:38:41.531 INFO:tasks.rados.rados.0.smithi138.stdout:242: expect (ObjNum 20 snap 0 seq_num 20) 2023-06-27T12:38:41.531 INFO:tasks.rados.rados.0.smithi138.stdout:243: snap_create 2023-06-27T12:38:41.531 INFO:tasks.rados.rados.0.smithi138.stdout:234: finishing write tid 2 to smithi138138488-14 2023-06-27T12:38:41.531 INFO:tasks.rados.rados.0.smithi138.stdout:234: finishing write tid 3 to smithi138138488-14 2023-06-27T12:38:41.531 INFO:tasks.rados.rados.0.smithi138.stdout:234: finishing write tid 4 to smithi138138488-14 2023-06-27T12:38:41.531 INFO:tasks.rados.rados.0.smithi138.stdout:234: oid 14 updating version 0 to 55 2023-06-27T12:38:41.531 INFO:tasks.rados.rados.0.smithi138.stdout:234: oid 14 version 55 is already newer than 55 2023-06-27T12:38:41.532 INFO:tasks.rados.rados.0.smithi138.stdout:234: oid 14 version 55 is already newer than 55 2023-06-27T12:38:41.532 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 55 (ObjNum 139 snap 17 seq_num 139) dirty exists 2023-06-27T12:38:41.532 INFO:tasks.rados.rados.0.smithi138.stdout:234: left oid 14 (ObjNum 139 snap 17 seq_num 139) 2023-06-27T12:38:41.532 INFO:tasks.rados.rados.0.smithi138.stdout:232: finishing write tid 2 to smithi138138488-2 2023-06-27T12:38:41.532 INFO:tasks.rados.rados.0.smithi138.stdout:232: finishing write tid 3 to smithi138138488-2 2023-06-27T12:38:41.532 INFO:tasks.rados.rados.0.smithi138.stdout:232: finishing write tid 4 to smithi138138488-2 2023-06-27T12:38:41.532 INFO:tasks.rados.rados.0.smithi138.stdout:232: oid 2 updating version 0 to 65 2023-06-27T12:38:41.532 INFO:tasks.rados.rados.0.smithi138.stdout:232: oid 2 version 65 is already newer than 65 2023-06-27T12:38:41.533 INFO:tasks.rados.rados.0.smithi138.stdout:232: oid 2 version 65 is already newer than 65 2023-06-27T12:38:41.533 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 65 (ObjNum 138 snap 17 seq_num 138) dirty exists 2023-06-27T12:38:41.533 INFO:tasks.rados.rados.0.smithi138.stdout:232: left oid 2 (ObjNum 138 snap 17 seq_num 138) 2023-06-27T12:38:41.582 INFO:tasks.rados.rados.0.smithi138.stdout:242: expect (ObjNum 20 snap 0 seq_num 20) 2023-06-27T12:38:42.537 INFO:tasks.rados.rados.0.smithi138.stdout:232: done (6 left) 2023-06-27T12:38:42.537 INFO:tasks.rados.rados.0.smithi138.stdout:234: done (5 left) 2023-06-27T12:38:42.537 INFO:tasks.rados.rados.0.smithi138.stdout:236: done (4 left) 2023-06-27T12:38:42.538 INFO:tasks.rados.rados.0.smithi138.stdout:239: done (3 left) 2023-06-27T12:38:42.538 INFO:tasks.rados.rados.0.smithi138.stdout:241: done (2 left) 2023-06-27T12:38:42.538 INFO:tasks.rados.rados.0.smithi138.stdout:242: done (1 left) 2023-06-27T12:38:42.538 INFO:tasks.rados.rados.0.smithi138.stdout:243: done (0 left) 2023-06-27T12:38:42.539 INFO:tasks.rados.rados.0.smithi138.stdout:244: write (excl) oid 49 current snap is 18 2023-06-27T12:38:42.539 INFO:tasks.rados.rados.0.smithi138.stdout:244: seq_num 142 ranges {786594=779712,2326443=60151} 2023-06-27T12:38:42.548 INFO:tasks.rados.rados.0.smithi138.stdout:244: writing smithi138138488-49 from 786594 to 1566306 tid 1 2023-06-27T12:38:42.553 INFO:tasks.rados.rados.0.smithi138.stdout:244: writing smithi138138488-49 from 2326443 to 2386594 tid 2 2023-06-27T12:38:42.553 INFO:tasks.rados.rados.0.smithi138.stdout:245: snap_create 2023-06-27T12:38:42.629 INFO:tasks.rados.rados.0.smithi138.stdout:244: finishing write tid 1 to smithi138138488-49 2023-06-27T12:38:42.711 INFO:tasks.rados.rados.0.smithi138.stdout:244: finishing write tid 2 to smithi138138488-49 2023-06-27T12:38:42.712 INFO:tasks.rados.rados.0.smithi138.stdout:244: finishing write tid 3 to smithi138138488-49 2023-06-27T12:38:42.713 INFO:tasks.rados.rados.0.smithi138.stdout:244: finishing write tid 4 to smithi138138488-49 2023-06-27T12:38:42.714 INFO:tasks.rados.rados.0.smithi138.stdout:244: oid 49 updating version 0 to 38 2023-06-27T12:38:42.714 INFO:tasks.rados.rados.0.smithi138.stdout:244: oid 49 version 38 is already newer than 36 2023-06-27T12:38:42.714 INFO:tasks.rados.rados.0.smithi138.stdout:244: oid 49 version 38 is already newer than 38 2023-06-27T12:38:42.715 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 49 v 38 (ObjNum 142 snap 18 seq_num 142) dirty exists 2023-06-27T12:38:42.715 INFO:tasks.rados.rados.0.smithi138.stdout:244: left oid 49 (ObjNum 142 snap 18 seq_num 142) 2023-06-27T12:38:42.717 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:42.852 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:42.847+0000 7f04464e6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:42.855 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:42.852+0000 7f04464e6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:43.542 INFO:tasks.rados.rados.0.smithi138.stdout:244: done (1 left) 2023-06-27T12:38:43.542 INFO:tasks.rados.rados.0.smithi138.stdout:245: done (0 left) 2023-06-27T12:38:43.542 INFO:tasks.rados.rados.0.smithi138.stdout:246: read oid 49 snap -1 2023-06-27T12:38:43.542 INFO:tasks.rados.rados.0.smithi138.stdout:246: expect (ObjNum 142 snap 18 seq_num 142) 2023-06-27T12:38:43.543 INFO:tasks.rados.rados.0.smithi138.stdout:247: write oid 37 current snap is 19 2023-06-27T12:38:43.543 INFO:tasks.rados.rados.0.smithi138.stdout:247: seq_num 143 ranges {403401=454961,1584925=506039} 2023-06-27T12:38:43.547 INFO:tasks.rados.rados.0.smithi138.stdout:247: writing smithi138138488-37 from 403401 to 858362 tid 1 2023-06-27T12:38:43.556 INFO:tasks.rados.rados.0.smithi138.stdout:247: writing smithi138138488-37 from 1584925 to 2090964 tid 2 2023-06-27T12:38:43.556 INFO:tasks.rados.rados.0.smithi138.stdout:248: write oid 48 current snap is 19 2023-06-27T12:38:43.556 INFO:tasks.rados.rados.0.smithi138.stdout:248: seq_num 144 ranges {420208=530210,1527054=766050} 2023-06-27T12:38:43.562 INFO:tasks.rados.rados.0.smithi138.stdout:248: writing smithi138138488-48 from 420208 to 950418 tid 1 2023-06-27T12:38:43.570 INFO:tasks.rados.rados.0.smithi138.stdout:248: writing smithi138138488-48 from 1527054 to 2293104 tid 2 2023-06-27T12:38:43.570 INFO:tasks.rados.rados.0.smithi138.stdout:249: snap_create 2023-06-27T12:38:43.592 INFO:tasks.rados.rados.0.smithi138.stdout:246: expect (ObjNum 142 snap 18 seq_num 142) 2023-06-27T12:38:43.688 INFO:tasks.rados.rados.0.smithi138.stdout:247: finishing write tid 1 to smithi138138488-37 2023-06-27T12:38:43.689 INFO:tasks.rados.rados.0.smithi138.stdout:248: finishing write tid 1 to smithi138138488-48 2023-06-27T12:38:43.689 INFO:tasks.rados.rados.0.smithi138.stdout:247: finishing write tid 2 to smithi138138488-37 2023-06-27T12:38:43.689 INFO:tasks.rados.rados.0.smithi138.stdout:247: finishing write tid 3 to smithi138138488-37 2023-06-27T12:38:43.689 INFO:tasks.rados.rados.0.smithi138.stdout:247: finishing write tid 4 to smithi138138488-37 2023-06-27T12:38:43.690 INFO:tasks.rados.rados.0.smithi138.stdout:247: oid 37 updating version 0 to 41 2023-06-27T12:38:43.690 INFO:tasks.rados.rados.0.smithi138.stdout:247: oid 37 version 41 is already newer than 41 2023-06-27T12:38:43.690 INFO:tasks.rados.rados.0.smithi138.stdout:247: oid 37 version 41 is already newer than 41 2023-06-27T12:38:43.690 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 41 (ObjNum 143 snap 19 seq_num 143) dirty exists 2023-06-27T12:38:43.690 INFO:tasks.rados.rados.0.smithi138.stdout:247: left oid 37 (ObjNum 143 snap 19 seq_num 143) 2023-06-27T12:38:43.690 INFO:tasks.rados.rados.0.smithi138.stdout:248: finishing write tid 2 to smithi138138488-48 2023-06-27T12:38:43.690 INFO:tasks.rados.rados.0.smithi138.stdout:248: finishing write tid 3 to smithi138138488-48 2023-06-27T12:38:43.690 INFO:tasks.rados.rados.0.smithi138.stdout:248: finishing write tid 4 to smithi138138488-48 2023-06-27T12:38:43.691 INFO:tasks.rados.rados.0.smithi138.stdout:248: oid 48 updating version 0 to 41 2023-06-27T12:38:43.691 INFO:tasks.rados.rados.0.smithi138.stdout:248: oid 48 version 41 is already newer than 41 2023-06-27T12:38:43.691 INFO:tasks.rados.rados.0.smithi138.stdout:248: oid 48 version 41 is already newer than 41 2023-06-27T12:38:43.691 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 41 (ObjNum 144 snap 19 seq_num 144) dirty exists 2023-06-27T12:38:43.691 INFO:tasks.rados.rados.0.smithi138.stdout:248: left oid 48 (ObjNum 144 snap 19 seq_num 144) 2023-06-27T12:38:44.549 INFO:tasks.rados.rados.0.smithi138.stdout:246: done (3 left) 2023-06-27T12:38:44.549 INFO:tasks.rados.rados.0.smithi138.stdout:247: done (2 left) 2023-06-27T12:38:44.549 INFO:tasks.rados.rados.0.smithi138.stdout:248: done (1 left) 2023-06-27T12:38:44.549 INFO:tasks.rados.rados.0.smithi138.stdout:249: done (0 left) 2023-06-27T12:38:44.550 INFO:tasks.rados.rados.0.smithi138.stdout:250: read oid 44 snap -1 2023-06-27T12:38:44.550 INFO:tasks.rados.rados.0.smithi138.stdout:250: expect (ObjNum 105 snap 9 seq_num 105) 2023-06-27T12:38:44.550 INFO:tasks.rados.rados.0.smithi138.stdout:251: write (excl) oid 32 current snap is 20 2023-06-27T12:38:44.550 INFO:tasks.rados.rados.0.smithi138.stdout:251: seq_num 145 ranges {437015=721812,1701889=509708} 2023-06-27T12:38:44.557 INFO:tasks.rados.rados.0.smithi138.stdout:251: writing smithi138138488-32 from 437015 to 1158827 tid 1 2023-06-27T12:38:44.565 INFO:tasks.rados.rados.0.smithi138.stdout:251: writing smithi138138488-32 from 1701889 to 2211597 tid 2 2023-06-27T12:38:44.565 INFO:tasks.rados.rados.0.smithi138.stdout:252: write oid 10 current snap is 20 2023-06-27T12:38:44.566 INFO:tasks.rados.rados.0.smithi138.stdout:252: seq_num 146 ranges {453822=797061,1760371=653366} 2023-06-27T12:38:44.573 INFO:tasks.rados.rados.0.smithi138.stdout:252: writing smithi138138488-10 from 453822 to 1250883 tid 1 2023-06-27T12:38:44.581 INFO:tasks.rados.rados.0.smithi138.stdout:252: writing smithi138138488-10 from 1760371 to 2413737 tid 2 2023-06-27T12:38:44.581 INFO:tasks.rados.rados.0.smithi138.stdout:253: delete oid 46 current snap is 20 2023-06-27T12:38:44.631 INFO:tasks.rados.rados.0.smithi138.stdout:253: done (3 left) 2023-06-27T12:38:44.631 INFO:tasks.rados.rados.0.smithi138.stdout:254: snap_remove snap 19 2023-06-27T12:38:44.927 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:45.054 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:45.049+0000 7facc5c45700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:45.057 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:45.053+0000 7facc5c45700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:45.548 INFO:tasks.rados.rados.0.smithi138.stdout:254: done (3 left) 2023-06-27T12:38:45.548 INFO:tasks.rados.rados.0.smithi138.stdout:255: delete oid 43 current snap is 20 2023-06-27T12:38:45.548 INFO:tasks.rados.rados.0.smithi138.stdout:250: expect (ObjNum 105 snap 9 seq_num 105) 2023-06-27T12:38:45.613 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:45.614 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:45.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:45.744 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:45.737+0000 7f23ccbbb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:45.747 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:45.743+0000 7f23ccbbb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:45.784 INFO:tasks.rados.rados.0.smithi138.stdout:251: finishing write tid 1 to smithi138138488-32 2023-06-27T12:38:45.784 INFO:tasks.rados.rados.0.smithi138.stdout:252: finishing write tid 1 to smithi138138488-10 2023-06-27T12:38:45.785 INFO:tasks.rados.rados.0.smithi138.stdout:251: finishing write tid 2 to smithi138138488-32 2023-06-27T12:38:45.785 INFO:tasks.rados.rados.0.smithi138.stdout:251: finishing write tid 3 to smithi138138488-32 2023-06-27T12:38:45.785 INFO:tasks.rados.rados.0.smithi138.stdout:251: finishing write tid 4 to smithi138138488-32 2023-06-27T12:38:45.785 INFO:tasks.rados.rados.0.smithi138.stdout:251: oid 32 updating version 0 to 77 2023-06-27T12:38:45.785 INFO:tasks.rados.rados.0.smithi138.stdout:251: oid 32 version 77 is already newer than 74 2023-06-27T12:38:45.785 INFO:tasks.rados.rados.0.smithi138.stdout:251: oid 32 version 77 is already newer than 77 2023-06-27T12:38:45.785 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 77 (ObjNum 145 snap 20 seq_num 145) dirty exists 2023-06-27T12:38:45.785 INFO:tasks.rados.rados.0.smithi138.stdout:251: left oid 32 (ObjNum 145 snap 20 seq_num 145) 2023-06-27T12:38:45.786 INFO:tasks.rados.rados.0.smithi138.stdout:252: finishing write tid 2 to smithi138138488-10 2023-06-27T12:38:45.786 INFO:tasks.rados.rados.0.smithi138.stdout:250: done (3 left) 2023-06-27T12:38:45.786 INFO:tasks.rados.rados.0.smithi138.stdout:251: done (2 left) 2023-06-27T12:38:45.786 INFO:tasks.rados.rados.0.smithi138.stdout:255: done (1 left) 2023-06-27T12:38:45.786 INFO:tasks.rados.rados.0.smithi138.stdout:256: read oid 50 snap 18 2023-06-27T12:38:45.786 INFO:tasks.rados.rados.0.smithi138.stdout:256: expect (ObjNum 79 snap 6 seq_num 79) 2023-06-27T12:38:45.786 INFO:tasks.rados.rados.0.smithi138.stdout:257: write (excl) oid 32 current snap is 20 2023-06-27T12:38:45.787 INFO:tasks.rados.rados.0.smithi138.stdout:252: finishing write tid 3 to smithi138138488-10 2023-06-27T12:38:45.787 INFO:tasks.rados.rados.0.smithi138.stdout:252: finishing write tid 4 to smithi138138488-10 2023-06-27T12:38:45.787 INFO:tasks.rados.rados.0.smithi138.stdout:252: oid 10 updating version 0 to 39 2023-06-27T12:38:45.787 INFO:tasks.rados.rados.0.smithi138.stdout:252: oid 10 version 39 is already newer than 39 2023-06-27T12:38:45.787 INFO:tasks.rados.rados.0.smithi138.stdout:252: oid 10 version 39 is already newer than 39 2023-06-27T12:38:45.787 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 39 (ObjNum 146 snap 20 seq_num 146) dirty exists 2023-06-27T12:38:45.787 INFO:tasks.rados.rados.0.smithi138.stdout:252: left oid 10 (ObjNum 146 snap 20 seq_num 146) 2023-06-27T12:38:45.787 INFO:tasks.rados.rados.0.smithi138.stdout:257: seq_num 149 ranges {504243=622808,1819464=517046} 2023-06-27T12:38:45.790 INFO:tasks.rados.rados.0.smithi138.stdout:257: writing smithi138138488-32 from 504243 to 1127051 tid 1 2023-06-27T12:38:45.798 INFO:tasks.rados.rados.0.smithi138.stdout:257: writing smithi138138488-32 from 1819464 to 2336510 tid 2 2023-06-27T12:38:45.799 INFO:tasks.rados.rados.0.smithi138.stdout:252: done (2 left) 2023-06-27T12:38:45.799 INFO:tasks.rados.rados.0.smithi138.stdout:258: write oid 29 current snap is 20 2023-06-27T12:38:45.799 INFO:tasks.rados.rados.0.smithi138.stdout:258: seq_num 150 ranges {521050=698057,1877946=643104} 2023-06-27T12:38:45.808 INFO:tasks.rados.rados.0.smithi138.stdout:258: writing smithi138138488-29 from 521050 to 1219107 tid 1 2023-06-27T12:38:45.818 INFO:tasks.rados.rados.0.smithi138.stdout:258: writing smithi138138488-29 from 1877946 to 2521050 tid 2 2023-06-27T12:38:45.818 INFO:tasks.rados.rados.0.smithi138.stdout:259: snap_create 2023-06-27T12:38:45.853 INFO:tasks.rados.rados.0.smithi138.stdout:256: expect (ObjNum 79 snap 6 seq_num 79) 2023-06-27T12:38:46.001 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:46.003 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:46.014 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2023-06-27T12:38:45.914459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112675459,"num_objects":88,"num_object_clones":38,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":541,"num_read_kb":269798,"num_write":647,"num_write_kb":147197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":609,"ondisk_log_size":609,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":388660,"kb_used_data":281364,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377098700,"statfs":{"total":386547056640,"available":386149068800,"internally_reserved":0,"allocated":288116736,"data_stored":304134918,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-184778,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":15636,"num_write":20,"num_write_kb":4994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000797"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-06-27T12:38:41.551021+0000","last_change":"2023-06-27T12:38:41.564569+0000","last_active":"2023-06-27T12:38:41.551021+0000","last_peered":"2023-06-27T12:38:41.551021+0000","last_clean":"2023-06-27T12:38:41.551021+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:41.551021+0000","last_undegraded":"2023-06-27T12:38:41.551021+0000","last_fullsized":"2023-06-27T12:38:41.551021+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"48'43","reported_seq":153,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.553185+0000","last_change":"2023-06-27T12:38:42.578050+0000","last_active":"2023-06-27T12:38:43.553185+0000","last_peered":"2023-06-27T12:38:43.553185+0000","last_clean":"2023-06-27T12:38:43.553185+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:43.553185+0000","last_undegraded":"2023-06-27T12:38:43.553185+0000","last_fullsized":"2023-06-27T12:38:43.553185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8184943,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":27,"num_read_kb":20813,"num_write":40,"num_write_kb":11426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'41","reported_seq":144,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.590931+0000","last_change":"2023-06-27T12:38:41.568002+0000","last_active":"2023-06-27T12:38:43.590931+0000","last_peered":"2023-06-27T12:38:43.590931+0000","last_clean":"2023-06-27T12:38:43.590931+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:43.590931+0000","last_undegraded":"2023-06-27T12:38:43.590931+0000","last_fullsized":"2023-06-27T12:38:43.590931+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5327786,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":12454,"num_write":43,"num_write_kb":11376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-06-27T12:38:41.560466+0000","last_change":"2023-06-27T12:38:41.581806+0000","last_active":"2023-06-27T12:38:41.560466+0000","last_peered":"2023-06-27T12:38:41.560466+0000","last_clean":"2023-06-27T12:38:41.560466+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:41.560466+0000","last_undegraded":"2023-06-27T12:38:41.560466+0000","last_fullsized":"2023-06-27T12:38:41.560466+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":23,"num_read_kb":3229,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"48'17","reported_seq":116,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.883622+0000","last_change":"2023-06-27T12:38:43.898638+0000","last_active":"2023-06-27T12:38:43.883622+0000","last_peered":"2023-06-27T12:38:43.883622+0000","last_clean":"2023-06-27T12:38:43.883622+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:43.883622+0000","last_undegraded":"2023-06-27T12:38:43.883622+0000","last_fullsized":"2023-06-27T12:38:43.883622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268978,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":14490,"num_write":21,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"51'50","reported_seq":159,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-06-27T12:38:41.551532+0000","last_change":"2023-06-27T12:38:41.617278+0000","last_active":"2023-06-27T12:38:41.551532+0000","last_peered":"2023-06-27T12:38:41.551532+0000","last_clean":"2023-06-27T12:38:41.551532+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:41.551532+0000","last_undegraded":"2023-06-27T12:38:41.551532+0000","last_fullsized":"2023-06-27T12:38:41.551532+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11630345,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":69,"num_read_kb":39220,"num_write":49,"num_write_kb":14021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"46'35","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-06-27T12:38:41.549184+0000","last_change":"2023-06-27T12:38:41.578656+0000","last_active":"2023-06-27T12:38:41.549184+0000","last_peered":"2023-06-27T12:38:41.549184+0000","last_clean":"2023-06-27T12:38:41.549184+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:41.549184+0000","last_undegraded":"2023-06-27T12:38:41.549184+0000","last_fullsized":"2023-06-27T12:38:41.549184+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731154,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":17109,"num_write":34,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"28'8","reported_seq":107,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.547357+0000","last_change":"2023-06-27T12:38:41.557571+0000","last_active":"2023-06-27T12:38:43.547357+0000","last_peered":"2023-06-27T12:38:43.547357+0000","last_clean":"2023-06-27T12:38:43.547357+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:43.547357+0000","last_undegraded":"2023-06-27T12:38:43.547357+0000","last_fullsized":"2023-06-27T12:38:43.547357+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3327753,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":11710,"num_write":9,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.553238+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:43.553238+0000","last_peered":"2023-06-27T12:38:43.553238+0000","last_clean":"2023-06-27T12:38:43.553238+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:43.553238+0000","last_undegraded":"2023-06-27T12:38:43.553238+0000","last_fullsized":"2023-06-27T12:38:43.553238+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":90,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.549987+0000","last_change":"2023-06-27T12:38:41.572916+0000","last_active":"2023-06-27T12:38:43.549987+0000","last_peered":"2023-06-27T12:38:43.549987+0000","last_clean":"2023-06-27T12:38:43.549987+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:43.549987+0000","last_undegraded":"2023-06-27T12:38:43.549987+0000","last_fullsized":"2023-06-27T12:38:43.549987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":71,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-06-27T12:38:41.544645+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:41.544645+0000","last_peered":"2023-06-27T12:38:41.544645+0000","last_clean":"2023-06-27T12:38:41.544645+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:41.544645+0000","last_undegraded":"2023-06-27T12:38:41.544645+0000","last_fullsized":"2023-06-27T12:38:41.544645+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"47'34","reported_seq":137,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.548346+0000","last_change":"2023-06-27T12:38:42.576831+0000","last_active":"2023-06-27T12:38:43.548346+0000","last_peered":"2023-06-27T12:38:43.548346+0000","last_clean":"2023-06-27T12:38:43.548346+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:43.548346+0000","last_undegraded":"2023-06-27T12:38:43.548346+0000","last_fullsized":"2023-06-27T12:38:43.548346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294172,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":18,"num_read_kb":10007,"num_write":29,"num_write_kb":7882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.552140+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:43.552140+0000","last_peered":"2023-06-27T12:38:43.552140+0000","last_clean":"2023-06-27T12:38:43.552140+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:43.552140+0000","last_undegraded":"2023-06-27T12:38:43.552140+0000","last_fullsized":"2023-06-27T12:38:43.552140+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"51'65","reported_seq":178,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.898364+0000","last_change":"2023-06-27T12:38:43.941551+0000","last_active":"2023-06-27T12:38:43.898364+0000","last_peered":"2023-06-27T12:38:43.898364+0000","last_clean":"2023-06-27T12:38:43.898364+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:43.898364+0000","last_undegraded":"2023-06-27T12:38:43.898364+0000","last_fullsized":"2023-06-27T12:38:43.898364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14084167,"num_objects":9,"num_object_clones":4,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":43537,"num_write":71,"num_write_kb":20803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":77,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.551724+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:43.551724+0000","last_peered":"2023-06-27T12:38:43.551724+0000","last_clean":"2023-06-27T12:38:43.551724+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:43.551724+0000","last_undegraded":"2023-06-27T12:38:43.551724+0000","last_fullsized":"2023-06-27T12:38:43.551724+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":204,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.895782+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:43.895782+0000","last_peered":"2023-06-27T12:38:43.895782+0000","last_clean":"2023-06-27T12:38:43.895782+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:43.895782+0000","last_undegraded":"2023-06-27T12:38:43.895782+0000","last_fullsized":"2023-06-27T12:38:43.895782+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"51'55","reported_seq":170,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.555431+0000","last_change":"2023-06-27T12:38:41.634988+0000","last_active":"2023-06-27T12:38:43.555431+0000","last_peered":"2023-06-27T12:38:43.555431+0000","last_clean":"2023-06-27T12:38:43.555431+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:43.555431+0000","last_undegraded":"2023-06-27T12:38:43.555431+0000","last_fullsized":"2023-06-27T12:38:43.555431+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12722800,"num_objects":9,"num_object_clones":3,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":66,"num_read_kb":27016,"num_write":57,"num_write_kb":15522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-06-27T12:38:41.559391+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:41.559391+0000","last_peered":"2023-06-27T12:38:41.559391+0000","last_clean":"2023-06-27T12:38:41.559391+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:41.559391+0000","last_undegraded":"2023-06-27T12:38:41.559391+0000","last_fullsized":"2023-06-27T12:38:41.559391+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"39'29","reported_seq":128,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.893205+0000","last_change":"2023-06-27T12:38:43.899076+0000","last_active":"2023-06-27T12:38:43.893205+0000","last_peered":"2023-06-27T12:38:43.893205+0000","last_clean":"2023-06-27T12:38:43.893205+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:43.893205+0000","last_undegraded":"2023-06-27T12:38:43.893205+0000","last_fullsized":"2023-06-27T12:38:43.893205+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798382,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.894088+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:43.894088+0000","last_peered":"2023-06-27T12:38:43.894088+0000","last_clean":"2023-06-27T12:38:43.894088+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:43.894088+0000","last_undegraded":"2023-06-27T12:38:43.894088+0000","last_fullsized":"2023-06-27T12:38:43.894088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"38'40","reported_seq":146,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.550983+0000","last_change":"2023-06-27T12:38:42.576343+0000","last_active":"2023-06-27T12:38:43.550983+0000","last_peered":"2023-06-27T12:38:43.550983+0000","last_clean":"2023-06-27T12:38:43.550983+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:43.550983+0000","last_undegraded":"2023-06-27T12:38:43.550983+0000","last_fullsized":"2023-06-27T12:38:43.550983+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13832029,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":15303,"num_write":41,"num_write_kb":12814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"50'14","reported_seq":109,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.891468+0000","last_change":"2023-06-27T12:38:43.947231+0000","last_active":"2023-06-27T12:38:43.891468+0000","last_peered":"2023-06-27T12:38:43.891468+0000","last_clean":"2023-06-27T12:38:43.891468+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:43.891468+0000","last_undegraded":"2023-06-27T12:38:43.891468+0000","last_fullsized":"2023-06-27T12:38:43.891468+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5798382,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":13725,"num_write":17,"num_write_kb":4552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.552421+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:43.552421+0000","last_peered":"2023-06-27T12:38:43.552421+0000","last_clean":"2023-06-27T12:38:43.552421+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:43.552421+0000","last_undegraded":"2023-06-27T12:38:43.552421+0000","last_fullsized":"2023-06-27T12:38:43.552421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":77,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.551859+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:43.551859+0000","last_peered":"2023-06-27T12:38:43.551859+0000","last_clean":"2023-06-27T12:38:43.551859+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:43.551859+0000","last_undegraded":"2023-06-27T12:38:43.551859+0000","last_fullsized":"2023-06-27T12:38:43.551859+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"55'41","reported_seq":148,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-06-27T12:38:43.600065+0000","last_change":"2023-06-27T12:38:42.577702+0000","last_active":"2023-06-27T12:38:43.600065+0000","last_peered":"2023-06-27T12:38:43.600065+0000","last_clean":"2023-06-27T12:38:43.600065+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:43.600065+0000","last_undegraded":"2023-06-27T12:38:43.600065+0000","last_fullsized":"2023-06-27T12:38:43.600065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7949678,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":22468,"num_write":35,"num_write_kb":10153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":112085091,"num_objects":86,"num_object_clones":38,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":438,"num_read_kb":269707,"num_write":510,"num_write_kb":145303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":512,"ondisk_log_size":512,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102024,"kb_used_data":75200,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94269816,"statfs":{"total":96636764160,"available":96532291584,"internally_reserved":0,"allocated":77004800,"data_stored":81071904,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97884,"kb_used_data":71060,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94273956,"statfs":{"total":96636764160,"available":96536530944,"internally_reserved":0,"allocated":72765440,"data_stored":76225869,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705715,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90328,"kb_used_data":63504,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94281512,"statfs":{"total":96636764160,"available":96544268288,"internally_reserved":0,"allocated":65028096,"data_stored":68982400,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98424,"kb_used_data":71600,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94273416,"statfs":{"total":96636764160,"available":96535977984,"internally_reserved":0,"allocated":73318400,"data_stored":77854745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:46.067 INFO:tasks.rados.rados.0.smithi138.stdout:257: finishing write tid 1 to smithi138138488-32 2023-06-27T12:38:46.067 INFO:tasks.rados.rados.0.smithi138.stdout:258: finishing write tid 1 to smithi138138488-29 2023-06-27T12:38:46.068 INFO:tasks.rados.rados.0.smithi138.stdout:258: finishing write tid 2 to smithi138138488-29 2023-06-27T12:38:46.068 INFO:tasks.rados.rados.0.smithi138.stdout:258: finishing write tid 3 to smithi138138488-29 2023-06-27T12:38:46.068 INFO:tasks.rados.rados.0.smithi138.stdout:257: finishing write tid 2 to smithi138138488-32 2023-06-27T12:38:46.068 INFO:tasks.rados.rados.0.smithi138.stdout:257: finishing write tid 3 to smithi138138488-32 2023-06-27T12:38:46.068 INFO:tasks.rados.rados.0.smithi138.stdout:257: finishing write tid 4 to smithi138138488-32 2023-06-27T12:38:46.068 INFO:tasks.rados.rados.0.smithi138.stdout:257: oid 32 updating version 0 to 82 2023-06-27T12:38:46.069 INFO:tasks.rados.rados.0.smithi138.stdout:257: oid 32 version 82 is already newer than 80 2023-06-27T12:38:46.069 INFO:tasks.rados.rados.0.smithi138.stdout:257: oid 32 version 82 is already newer than 82 2023-06-27T12:38:46.069 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 82 (ObjNum 149 snap 20 seq_num 149) dirty exists 2023-06-27T12:38:46.069 INFO:tasks.rados.rados.0.smithi138.stdout:257: left oid 32 (ObjNum 149 snap 20 seq_num 149) 2023-06-27T12:38:46.069 INFO:tasks.rados.rados.0.smithi138.stdout:258: finishing write tid 4 to smithi138138488-29 2023-06-27T12:38:46.069 INFO:tasks.rados.rados.0.smithi138.stdout:258: oid 29 updating version 0 to 24 2023-06-27T12:38:46.069 INFO:tasks.rados.rados.0.smithi138.stdout:258: oid 29 version 24 is already newer than 24 2023-06-27T12:38:46.070 INFO:tasks.rados.rados.0.smithi138.stdout:258: oid 29 version 24 is already newer than 24 2023-06-27T12:38:46.070 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 24 (ObjNum 150 snap 20 seq_num 150) dirty exists 2023-06-27T12:38:46.070 INFO:tasks.rados.rados.0.smithi138.stdout:258: left oid 29 (ObjNum 150 snap 20 seq_num 150) 2023-06-27T12:38:46.556 INFO:tasks.rados.rados.0.smithi138.stdout:256: done (3 left) 2023-06-27T12:38:46.557 INFO:tasks.rados.rados.0.smithi138.stdout:257: done (2 left) 2023-06-27T12:38:46.557 INFO:tasks.rados.rados.0.smithi138.stdout:258: done (1 left) 2023-06-27T12:38:46.557 INFO:tasks.rados.rados.0.smithi138.stdout:259: done (0 left) 2023-06-27T12:38:46.558 INFO:tasks.rados.rados.0.smithi138.stdout:260: write (excl) oid 45 current snap is 21 2023-06-27T12:38:46.558 INFO:tasks.rados.rados.0.smithi138.stdout:260: seq_num 151 ranges {537857=773306,1936428=520715} 2023-06-27T12:38:46.563 INFO:tasks.rados.rados.0.smithi138.stdout:260: writing smithi138138488-45 from 537857 to 1311163 tid 1 2023-06-27T12:38:46.569 INFO:tasks.rados.rados.0.smithi138.stdout:260: writing smithi138138488-45 from 1936428 to 2457143 tid 2 2023-06-27T12:38:46.569 INFO:tasks.rados.rados.0.smithi138.stdout:261: snap_remove snap 17 2023-06-27T12:38:47.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:47.255 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:47.250+0000 7f149b5bc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:47.258 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:47.254+0000 7f149b5bc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:47.557 INFO:tasks.rados.rados.0.smithi138.stdout:261: done (1 left) 2023-06-27T12:38:47.558 INFO:tasks.rados.rados.0.smithi138.stdout:262: write (excl) oid 9 current snap is 21 2023-06-27T12:38:47.558 INFO:tasks.rados.rados.0.smithi138.stdout:260: finishing write tid 1 to smithi138138488-45 2023-06-27T12:38:47.558 INFO:tasks.rados.rados.0.smithi138.stdout:262: seq_num 152 ranges {554664=448555,1478557=664373} 2023-06-27T12:38:47.564 INFO:tasks.rados.rados.0.smithi138.stdout:262: writing smithi138138488-9 from 554664 to 1003219 tid 1 2023-06-27T12:38:47.573 INFO:tasks.rados.rados.0.smithi138.stdout:262: writing smithi138138488-9 from 1478557 to 2142930 tid 2 2023-06-27T12:38:47.573 INFO:tasks.rados.rados.0.smithi138.stdout:263: write (excl) oid 20 current snap is 21 2023-06-27T12:38:47.573 INFO:tasks.rados.rados.0.smithi138.stdout:263: seq_num 153 ranges {571471=640157,1653392=524384} 2023-06-27T12:38:47.582 INFO:tasks.rados.rados.0.smithi138.stdout:263: writing smithi138138488-20 from 571471 to 1211628 tid 1 2023-06-27T12:38:47.588 INFO:tasks.rados.rados.0.smithi138.stdout:263: writing smithi138138488-20 from 1653392 to 2177776 tid 2 2023-06-27T12:38:47.588 INFO:tasks.rados.rados.0.smithi138.stdout:264: snap_create 2023-06-27T12:38:47.588 INFO:tasks.rados.rados.0.smithi138.stdout:260: finishing write tid 2 to smithi138138488-45 2023-06-27T12:38:47.588 INFO:tasks.rados.rados.0.smithi138.stdout:260: finishing write tid 3 to smithi138138488-45 2023-06-27T12:38:47.588 INFO:tasks.rados.rados.0.smithi138.stdout:260: finishing write tid 4 to smithi138138488-45 2023-06-27T12:38:47.588 INFO:tasks.rados.rados.0.smithi138.stdout:260: oid 45 updating version 0 to 83 2023-06-27T12:38:47.588 INFO:tasks.rados.rados.0.smithi138.stdout:260: oid 45 updating version 83 to 86 2023-06-27T12:38:47.589 INFO:tasks.rados.rados.0.smithi138.stdout:260: oid 45 version 86 is already newer than 86 2023-06-27T12:38:47.589 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 86 (ObjNum 151 snap 21 seq_num 151) dirty exists 2023-06-27T12:38:47.589 INFO:tasks.rados.rados.0.smithi138.stdout:260: left oid 45 (ObjNum 151 snap 21 seq_num 151) 2023-06-27T12:38:47.641 INFO:tasks.rados.rados.0.smithi138.stdout:262: finishing write tid 1 to smithi138138488-9 2023-06-27T12:38:47.659 INFO:tasks.rados.rados.0.smithi138.stdout:263: finishing write tid 1 to smithi138138488-20 2023-06-27T12:38:47.677 INFO:tasks.rados.rados.0.smithi138.stdout:263: finishing write tid 2 to smithi138138488-20 2023-06-27T12:38:47.696 INFO:tasks.rados.rados.0.smithi138.stdout:262: finishing write tid 2 to smithi138138488-9 2023-06-27T12:38:47.697 INFO:tasks.rados.rados.0.smithi138.stdout:262: finishing write tid 3 to smithi138138488-9 2023-06-27T12:38:47.697 INFO:tasks.rados.rados.0.smithi138.stdout:262: finishing write tid 4 to smithi138138488-9 2023-06-27T12:38:47.698 INFO:tasks.rados.rados.0.smithi138.stdout:262: oid 9 updating version 0 to 63 2023-06-27T12:38:47.698 INFO:tasks.rados.rados.0.smithi138.stdout:262: oid 9 version 63 is already newer than 63 2023-06-27T12:38:47.698 INFO:tasks.rados.rados.0.smithi138.stdout:262: oid 9 version 63 is already newer than 60 2023-06-27T12:38:47.698 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 63 (ObjNum 152 snap 21 seq_num 152) dirty exists 2023-06-27T12:38:47.698 INFO:tasks.rados.rados.0.smithi138.stdout:262: left oid 9 (ObjNum 152 snap 21 seq_num 152) 2023-06-27T12:38:47.719 INFO:tasks.rados.rados.0.smithi138.stdout:263: finishing write tid 3 to smithi138138488-20 2023-06-27T12:38:47.719 INFO:tasks.rados.rados.0.smithi138.stdout:263: finishing write tid 4 to smithi138138488-20 2023-06-27T12:38:47.720 INFO:tasks.rados.rados.0.smithi138.stdout:263: oid 20 updating version 0 to 44 2023-06-27T12:38:47.720 INFO:tasks.rados.rados.0.smithi138.stdout:263: oid 20 updating version 44 to 47 2023-06-27T12:38:47.720 INFO:tasks.rados.rados.0.smithi138.stdout:263: oid 20 version 47 is already newer than 47 2023-06-27T12:38:47.720 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 47 (ObjNum 153 snap 21 seq_num 153) dirty exists 2023-06-27T12:38:47.720 INFO:tasks.rados.rados.0.smithi138.stdout:263: left oid 20 (ObjNum 153 snap 21 seq_num 153) 2023-06-27T12:38:48.577 INFO:tasks.rados.rados.0.smithi138.stdout:260: done (3 left) 2023-06-27T12:38:48.578 INFO:tasks.rados.rados.0.smithi138.stdout:262: done (2 left) 2023-06-27T12:38:48.578 INFO:tasks.rados.rados.0.smithi138.stdout:263: done (1 left) 2023-06-27T12:38:48.578 INFO:tasks.rados.rados.0.smithi138.stdout:264: done (0 left) 2023-06-27T12:38:48.578 INFO:tasks.rados.rados.0.smithi138.stdout:265: snap_create 2023-06-27T12:38:49.329 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:49.467 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:49.463+0000 7f5714670700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:49.470 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:49.467+0000 7f5714670700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:49.578 INFO:tasks.rados.rados.0.smithi138.stdout:265: done (0 left) 2023-06-27T12:38:49.578 INFO:tasks.rados.rados.0.smithi138.stdout:266: snap_remove snap 22 2023-06-27T12:38:50.580 INFO:tasks.rados.rados.0.smithi138.stdout:266: done (0 left) 2023-06-27T12:38:50.580 INFO:tasks.rados.rados.0.smithi138.stdout:267: write (excl) oid 9 current snap is 23 2023-06-27T12:38:50.580 INFO:tasks.rados.rados.0.smithi138.stdout:267: seq_num 154 ranges {588278=715406,1711874=668042} 2023-06-27T12:38:50.588 INFO:tasks.rados.rados.0.smithi138.stdout:267: writing smithi138138488-9 from 588278 to 1303684 tid 1 2023-06-27T12:38:50.594 INFO:tasks.rados.rados.0.smithi138.stdout:267: writing smithi138138488-9 from 1711874 to 2379916 tid 2 2023-06-27T12:38:50.595 INFO:tasks.rados.rados.0.smithi138.stdout:268: snap_create 2023-06-27T12:38:50.720 INFO:tasks.rados.rados.0.smithi138.stdout:267: finishing write tid 1 to smithi138138488-9 2023-06-27T12:38:50.771 INFO:tasks.rados.rados.0.smithi138.stdout:267: finishing write tid 2 to smithi138138488-9 2023-06-27T12:38:50.772 INFO:tasks.rados.rados.0.smithi138.stdout:267: finishing write tid 3 to smithi138138488-9 2023-06-27T12:38:50.773 INFO:tasks.rados.rados.0.smithi138.stdout:267: finishing write tid 4 to smithi138138488-9 2023-06-27T12:38:50.774 INFO:tasks.rados.rados.0.smithi138.stdout:267: oid 9 updating version 0 to 64 2023-06-27T12:38:50.774 INFO:tasks.rados.rados.0.smithi138.stdout:267: oid 9 updating version 64 to 67 2023-06-27T12:38:50.774 INFO:tasks.rados.rados.0.smithi138.stdout:267: oid 9 version 67 is already newer than 67 2023-06-27T12:38:50.774 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 67 (ObjNum 154 snap 23 seq_num 154) dirty exists 2023-06-27T12:38:50.774 INFO:tasks.rados.rados.0.smithi138.stdout:267: left oid 9 (ObjNum 154 snap 23 seq_num 154) 2023-06-27T12:38:51.017 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:51.017 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:51.018 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:51.146 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:51.139+0000 7faa8446e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:51.149 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:51.146+0000 7faa8446e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:51.342 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:51.344 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-06-27T12:38:50.574412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110305837,"num_objects":95,"num_object_clones":45,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":9,"num_read":568,"num_read_kb":284815,"num_write":675,"num_write_kb":155735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":660,"ondisk_log_size":660,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":399396,"kb_used_data":291116,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377087964,"statfs":{"total":386547056640,"available":386138075136,"internally_reserved":0,"allocated":298102784,"data_stored":311845306,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2369622,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":27,"num_read_kb":15017,"num_write":28,"num_write_kb":8538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022756"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":98,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.804922+0000","last_change":"2023-06-27T12:38:45.820374+0000","last_active":"2023-06-27T12:38:46.804922+0000","last_peered":"2023-06-27T12:38:46.804922+0000","last_clean":"2023-06-27T12:38:46.804922+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:46.804922+0000","last_undegraded":"2023-06-27T12:38:46.804922+0000","last_fullsized":"2023-06-27T12:38:46.804922+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"59'47","reported_seq":169,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-06-27T12:38:48.587905+0000","last_change":"2023-06-27T12:38:47.620352+0000","last_active":"2023-06-27T12:38:48.587905+0000","last_peered":"2023-06-27T12:38:48.587905+0000","last_clean":"2023-06-27T12:38:48.587905+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:48.587905+0000","last_undegraded":"2023-06-27T12:38:48.587905+0000","last_fullsized":"2023-06-27T12:38:48.587905+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8537890,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":29,"num_read_kb":20814,"num_write":44,"num_write_kb":12565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'41","reported_seq":154,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-06-27T12:38:47.563164+0000","last_change":"2023-06-27T12:38:47.649711+0000","last_active":"2023-06-27T12:38:47.563164+0000","last_peered":"2023-06-27T12:38:47.563164+0000","last_clean":"2023-06-27T12:38:47.563164+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:47.563164+0000","last_undegraded":"2023-06-27T12:38:47.563164+0000","last_fullsized":"2023-06-27T12:38:47.563164+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5327786,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":12454,"num_write":43,"num_write_kb":11376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":132,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.802257+0000","last_change":"2023-06-27T12:38:45.816421+0000","last_active":"2023-06-27T12:38:46.802257+0000","last_peered":"2023-06-27T12:38:46.802257+0000","last_clean":"2023-06-27T12:38:46.802257+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:46.802257+0000","last_undegraded":"2023-06-27T12:38:46.802257+0000","last_fullsized":"2023-06-27T12:38:46.802257+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":23,"num_read_kb":3229,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"48'17","reported_seq":123,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.586943+0000","last_change":"2023-06-27T12:38:45.578436+0000","last_active":"2023-06-27T12:38:46.586943+0000","last_peered":"2023-06-27T12:38:46.586943+0000","last_clean":"2023-06-27T12:38:46.586943+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:46.586943+0000","last_undegraded":"2023-06-27T12:38:46.586943+0000","last_fullsized":"2023-06-27T12:38:46.586943+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268978,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":14490,"num_write":21,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"51'50","reported_seq":171,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.801441+0000","last_change":"2023-06-27T12:38:45.816903+0000","last_active":"2023-06-27T12:38:46.801441+0000","last_peered":"2023-06-27T12:38:46.801441+0000","last_clean":"2023-06-27T12:38:46.801441+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:46.801441+0000","last_undegraded":"2023-06-27T12:38:46.801441+0000","last_fullsized":"2023-06-27T12:38:46.801441+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11630345,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":76,"num_read_kb":47698,"num_write":49,"num_write_kb":14021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"56'39","reported_seq":144,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.815633+0000","last_change":"2023-06-27T12:38:45.889490+0000","last_active":"2023-06-27T12:38:46.815633+0000","last_peered":"2023-06-27T12:38:46.815633+0000","last_clean":"2023-06-27T12:38:46.815633+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:46.815633+0000","last_undegraded":"2023-06-27T12:38:46.815633+0000","last_fullsized":"2023-06-27T12:38:46.815633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8033713,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":17110,"num_write":38,"num_write_kb":11366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"28'8","reported_seq":120,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-06-27T12:38:47.564585+0000","last_change":"2023-06-27T12:38:47.569731+0000","last_active":"2023-06-27T12:38:47.564585+0000","last_peered":"2023-06-27T12:38:47.564585+0000","last_clean":"2023-06-27T12:38:47.564585+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:47.564585+0000","last_undegraded":"2023-06-27T12:38:47.564585+0000","last_fullsized":"2023-06-27T12:38:47.564585+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3327753,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":18242,"num_write":9,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":95,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-06-27T12:38:48.583017+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:48.583017+0000","last_peered":"2023-06-27T12:38:48.583017+0000","last_clean":"2023-06-27T12:38:48.583017+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:48.583017+0000","last_undegraded":"2023-06-27T12:38:48.583017+0000","last_fullsized":"2023-06-27T12:38:48.583017+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":100,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-06-27T12:38:47.566613+0000","last_change":"2023-06-27T12:38:47.572014+0000","last_active":"2023-06-27T12:38:47.566613+0000","last_peered":"2023-06-27T12:38:47.566613+0000","last_clean":"2023-06-27T12:38:47.566613+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:47.566613+0000","last_undegraded":"2023-06-27T12:38:47.566613+0000","last_fullsized":"2023-06-27T12:38:47.566613+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.806516+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:46.806516+0000","last_peered":"2023-06-27T12:38:46.806516+0000","last_clean":"2023-06-27T12:38:46.806516+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:46.806516+0000","last_undegraded":"2023-06-27T12:38:46.806516+0000","last_fullsized":"2023-06-27T12:38:46.806516+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"47'34","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-06-27T12:38:48.586971+0000","last_change":"2023-06-27T12:38:47.621676+0000","last_active":"2023-06-27T12:38:48.586971+0000","last_peered":"2023-06-27T12:38:48.586971+0000","last_clean":"2023-06-27T12:38:48.586971+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:48.586971+0000","last_undegraded":"2023-06-27T12:38:48.586971+0000","last_fullsized":"2023-06-27T12:38:48.586971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294172,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":19,"num_read_kb":10007,"num_write":29,"num_write_kb":7882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-06-27T12:38:47.563807+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:47.563807+0000","last_peered":"2023-06-27T12:38:47.563807+0000","last_clean":"2023-06-27T12:38:47.563807+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:47.563807+0000","last_undegraded":"2023-06-27T12:38:47.563807+0000","last_fullsized":"2023-06-27T12:38:47.563807+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"58'86","reported_seq":197,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.648453+0000","last_change":"2023-06-27T12:38:45.630924+0000","last_active":"2023-06-27T12:38:46.648453+0000","last_peered":"2023-06-27T12:38:46.648453+0000","last_clean":"2023-06-27T12:38:46.648453+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:46.648453+0000","last_undegraded":"2023-06-27T12:38:46.648453+0000","last_fullsized":"2023-06-27T12:38:46.648453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336305,"num_objects":11,"num_object_clones":6,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":43540,"num_write":83,"num_write_kb":24385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-06-27T12:38:47.570413+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:47.570413+0000","last_peered":"2023-06-27T12:38:47.570413+0000","last_clean":"2023-06-27T12:38:47.570413+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:47.570413+0000","last_undegraded":"2023-06-27T12:38:47.570413+0000","last_fullsized":"2023-06-27T12:38:47.570413+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":210,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.581517+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:46.581517+0000","last_peered":"2023-06-27T12:38:46.581517+0000","last_clean":"2023-06-27T12:38:46.581517+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:46.581517+0000","last_undegraded":"2023-06-27T12:38:46.581517+0000","last_fullsized":"2023-06-27T12:38:46.581517+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"59'63","reported_seq":184,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-06-27T12:38:47.648880+0000","last_change":"2023-06-27T12:38:47.573120+0000","last_active":"2023-06-27T12:38:47.648880+0000","last_peered":"2023-06-27T12:38:47.648880+0000","last_clean":"2023-06-27T12:38:47.648880+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:47.648880+0000","last_undegraded":"2023-06-27T12:38:47.648880+0000","last_fullsized":"2023-06-27T12:38:47.648880+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13143008,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":68,"num_read_kb":27017,"num_write":61,"num_write_kb":16610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.810058+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:46.810058+0000","last_peered":"2023-06-27T12:38:46.810058+0000","last_clean":"2023-06-27T12:38:46.810058+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:46.810058+0000","last_undegraded":"2023-06-27T12:38:46.810058+0000","last_fullsized":"2023-06-27T12:38:46.810058+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"39'29","reported_seq":135,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.588040+0000","last_change":"2023-06-27T12:38:45.581155+0000","last_active":"2023-06-27T12:38:46.588040+0000","last_peered":"2023-06-27T12:38:46.588040+0000","last_clean":"2023-06-27T12:38:46.588040+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:46.588040+0000","last_undegraded":"2023-06-27T12:38:46.588040+0000","last_fullsized":"2023-06-27T12:38:46.588040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798382,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.583439+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:46.583439+0000","last_peered":"2023-06-27T12:38:46.583439+0000","last_clean":"2023-06-27T12:38:46.583439+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:46.583439+0000","last_undegraded":"2023-06-27T12:38:46.583439+0000","last_fullsized":"2023-06-27T12:38:46.583439+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"57'44","reported_seq":160,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-06-27T12:38:48.588423+0000","last_change":"2023-06-27T12:38:47.617544+0000","last_active":"2023-06-27T12:38:48.588423+0000","last_peered":"2023-06-27T12:38:48.588423+0000","last_clean":"2023-06-27T12:38:48.588423+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:48.588423+0000","last_undegraded":"2023-06-27T12:38:48.588423+0000","last_fullsized":"2023-06-27T12:38:48.588423+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11126135,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":15303,"num_write":41,"num_write_kb":12814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"57'24","reported_seq":120,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-06-27T12:38:46.588694+0000","last_change":"2023-06-27T12:38:45.581383+0000","last_active":"2023-06-27T12:38:46.588694+0000","last_peered":"2023-06-27T12:38:46.588694+0000","last_clean":"2023-06-27T12:38:46.588694+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:46.588694+0000","last_undegraded":"2023-06-27T12:38:46.588694+0000","last_fullsized":"2023-06-27T12:38:46.588694+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4806802,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":13726,"num_write":21,"num_write_kb":5863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":95,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-06-27T12:38:48.586823+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:48.586823+0000","last_peered":"2023-06-27T12:38:48.586823+0000","last_clean":"2023-06-27T12:38:48.586823+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:48.586823+0000","last_undegraded":"2023-06-27T12:38:48.586823+0000","last_fullsized":"2023-06-27T12:38:48.586823+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-06-27T12:38:47.566368+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:47.566368+0000","last_peered":"2023-06-27T12:38:47.566368+0000","last_clean":"2023-06-27T12:38:47.566368+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:47.566368+0000","last_undegraded":"2023-06-27T12:38:47.566368+0000","last_fullsized":"2023-06-27T12:38:47.566368+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"55'41","reported_seq":160,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-06-27T12:38:48.587714+0000","last_change":"2023-06-27T12:38:47.611203+0000","last_active":"2023-06-27T12:38:48.587714+0000","last_peered":"2023-06-27T12:38:48.587714+0000","last_clean":"2023-06-27T12:38:48.587714+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:48.587714+0000","last_undegraded":"2023-06-27T12:38:48.587714+0000","last_fullsized":"2023-06-27T12:38:48.587714+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7949678,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":22468,"num_write":35,"num_write_kb":10153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":109715469,"num_objects":93,"num_object_clones":45,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":9,"num_read":465,"num_read_kb":284724,"num_write":538,"num_write_kb":153841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":563,"ondisk_log_size":563,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103844,"kb_used_data":77020,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94267996,"statfs":{"total":96636764160,"available":96530427904,"internally_reserved":0,"allocated":78868480,"data_stored":82095481,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":99180,"kb_used_data":72356,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94272660,"statfs":{"total":96636764160,"available":96535203840,"internally_reserved":0,"allocated":74092544,"data_stored":76993277,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705720,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93724,"kb_used_data":66900,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94278116,"statfs":{"total":96636764160,"available":96540790784,"internally_reserved":0,"allocated":68505600,"data_stored":72340091,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":102648,"kb_used_data":74840,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94269192,"statfs":{"total":96636764160,"available":96531652608,"internally_reserved":0,"allocated":76636160,"data_stored":80416457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:51.542 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:51.586 INFO:tasks.rados.rados.0.smithi138.stdout:267: done (1 left) 2023-06-27T12:38:51.587 INFO:tasks.rados.rados.0.smithi138.stdout:268: done (0 left) 2023-06-27T12:38:51.587 INFO:tasks.rados.rados.0.smithi138.stdout:269: read oid 33 snap -1 2023-06-27T12:38:51.587 INFO:tasks.rados.rados.0.smithi138.stdout:269: expect (ObjNum 124 snap 15 seq_num 124) 2023-06-27T12:38:51.587 INFO:tasks.rados.rados.0.smithi138.stdout:270: delete oid 39 current snap is 24 2023-06-27T12:38:51.661 INFO:tasks.rados.rados.0.smithi138.stdout:270: done (1 left) 2023-06-27T12:38:51.661 INFO:tasks.rados.rados.0.smithi138.stdout:271: snap_remove snap 21 2023-06-27T12:38:51.691 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:51.686+0000 7ff439714700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:51.694 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:51.690+0000 7ff439714700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:52.593 INFO:tasks.rados.rados.0.smithi138.stdout:271: done (1 left) 2023-06-27T12:38:52.593 INFO:tasks.rados.rados.0.smithi138.stdout:272: write (excl) oid 31 current snap is 24 2023-06-27T12:38:52.593 INFO:tasks.rados.rados.0.smithi138.stdout:272: seq_num 156 ranges {621892=465904,1712485=671711} 2023-06-27T12:38:52.599 INFO:tasks.rados.rados.0.smithi138.stdout:272: writing smithi138138488-31 from 621892 to 1087796 tid 1 2023-06-27T12:38:52.608 INFO:tasks.rados.rados.0.smithi138.stdout:272: writing smithi138138488-31 from 1712485 to 2384196 tid 2 2023-06-27T12:38:52.608 INFO:tasks.rados.rados.0.smithi138.stdout:273: delete oid 28 current snap is 24 2023-06-27T12:38:52.608 INFO:tasks.rados.rados.0.smithi138.stdout:269: expect (ObjNum 124 snap 15 seq_num 124) 2023-06-27T12:38:52.734 INFO:tasks.rados.rados.0.smithi138.stdout:272: finishing write tid 1 to smithi138138488-31 2023-06-27T12:38:52.735 INFO:tasks.rados.rados.0.smithi138.stdout:269: done (2 left) 2023-06-27T12:38:52.735 INFO:tasks.rados.rados.0.smithi138.stdout:273: done (1 left) 2023-06-27T12:38:52.735 INFO:tasks.rados.rados.0.smithi138.stdout:274: delete oid 42 current snap is 24 2023-06-27T12:38:52.749 INFO:tasks.rados.rados.0.smithi138.stdout:274: done (1 left) 2023-06-27T12:38:52.750 INFO:tasks.rados.rados.0.smithi138.stdout:275: read oid 26 snap -1 2023-06-27T12:38:52.750 INFO:tasks.rados.rados.0.smithi138.stdout:275: expect (ObjNum 101 snap 9 seq_num 101) 2023-06-27T12:38:52.750 INFO:tasks.rados.rados.0.smithi138.stdout:276: snap_remove snap 20 2023-06-27T12:38:53.594 INFO:tasks.rados.rados.0.smithi138.stdout:276: done (2 left) 2023-06-27T12:38:53.594 INFO:tasks.rados.rados.0.smithi138.stdout:277: delete oid 32 current snap is 24 2023-06-27T12:38:53.594 INFO:tasks.rados.rados.0.smithi138.stdout:275: expect (ObjNum 101 snap 9 seq_num 101) 2023-06-27T12:38:53.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:53.806 INFO:tasks.rados.rados.0.smithi138.stdout:272: finishing write tid 2 to smithi138138488-31 2023-06-27T12:38:53.807 INFO:tasks.rados.rados.0.smithi138.stdout:275: done (2 left) 2023-06-27T12:38:53.807 INFO:tasks.rados.rados.0.smithi138.stdout:277: done (1 left) 2023-06-27T12:38:53.807 INFO:tasks.rados.rados.0.smithi138.stdout:278: write (excl) oid 37 current snap is 24 2023-06-27T12:38:53.807 INFO:tasks.rados.rados.0.smithi138.stdout:272: finishing write tid 3 to smithi138138488-31 2023-06-27T12:38:53.807 INFO:tasks.rados.rados.0.smithi138.stdout:272: finishing write tid 4 to smithi138138488-31 2023-06-27T12:38:53.807 INFO:tasks.rados.rados.0.smithi138.stdout:272: oid 31 updating version 0 to 58 2023-06-27T12:38:53.807 INFO:tasks.rados.rados.0.smithi138.stdout:272: oid 31 version 58 is already newer than 55 2023-06-27T12:38:53.808 INFO:tasks.rados.rados.0.smithi138.stdout:272: oid 31 version 58 is already newer than 58 2023-06-27T12:38:53.808 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 58 (ObjNum 156 snap 24 seq_num 156) dirty exists 2023-06-27T12:38:53.808 INFO:tasks.rados.rados.0.smithi138.stdout:272: left oid 31 (ObjNum 156 snap 24 seq_num 156) 2023-06-27T12:38:53.808 INFO:tasks.rados.rados.0.smithi138.stdout:278: seq_num 160 ranges {689120=483253,1946413=679049} 2023-06-27T12:38:53.812 INFO:tasks.rados.rados.0.smithi138.stdout:278: writing smithi138138488-37 from 689120 to 1172373 tid 1 2023-06-27T12:38:53.824 INFO:tasks.rados.rados.0.smithi138.stdout:278: writing smithi138138488-37 from 1946413 to 2625462 tid 2 2023-06-27T12:38:53.824 INFO:tasks.rados.rados.0.smithi138.stdout:272: done (1 left) 2023-06-27T12:38:53.825 INFO:tasks.rados.rados.0.smithi138.stdout:279: write (excl) oid 50 current snap is 24 2023-06-27T12:38:53.825 INFO:tasks.rados.rados.0.smithi138.stdout:279: seq_num 161 ranges {705927=558502,2004895=422707} 2023-06-27T12:38:53.833 INFO:tasks.rados.rados.0.smithi138.stdout:279: writing smithi138138488-50 from 705927 to 1264429 tid 1 2023-06-27T12:38:53.842 INFO:tasks.rados.rados.0.smithi138.stdout:279: writing smithi138138488-50 from 2004895 to 2427602 tid 2 2023-06-27T12:38:53.842 INFO:tasks.rados.rados.0.smithi138.stdout:280: snap_remove snap 18 2023-06-27T12:38:53.909 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:53.903+0000 7fef6b69f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:53.912 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:53.908+0000 7fef6b69f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:54.601 INFO:tasks.rados.rados.0.smithi138.stdout:280: done (2 left) 2023-06-27T12:38:54.601 INFO:tasks.rados.rados.0.smithi138.stdout:281: write (excl) oid 6 current snap is 24 2023-06-27T12:38:54.601 INFO:tasks.rados.rados.0.smithi138.stdout:278: finishing write tid 1 to smithi138138488-37 2023-06-27T12:38:54.601 INFO:tasks.rados.rados.0.smithi138.stdout:281: seq_num 162 ranges {722734=633751,2063377=659357} 2023-06-27T12:38:54.608 INFO:tasks.rados.rados.0.smithi138.stdout:281: writing smithi138138488-6 from 722734 to 1356485 tid 1 2023-06-27T12:38:54.615 INFO:tasks.rados.rados.0.smithi138.stdout:281: writing smithi138138488-6 from 2063377 to 2722734 tid 2 2023-06-27T12:38:54.616 INFO:tasks.rados.rados.0.smithi138.stdout:282: write oid 7 current snap is 24 2023-06-27T12:38:54.616 INFO:tasks.rados.rados.0.smithi138.stdout:278: finishing write tid 2 to smithi138138488-37 2023-06-27T12:38:54.616 INFO:tasks.rados.rados.0.smithi138.stdout:279: finishing write tid 1 to smithi138138488-50 2023-06-27T12:38:54.616 INFO:tasks.rados.rados.0.smithi138.stdout:282: seq_num 163 ranges {739541=709000,2121859=426376} 2023-06-27T12:38:54.623 INFO:tasks.rados.rados.0.smithi138.stdout:282: writing smithi138138488-7 from 739541 to 1448541 tid 1 2023-06-27T12:38:54.632 INFO:tasks.rados.rados.0.smithi138.stdout:282: writing smithi138138488-7 from 2121859 to 2548235 tid 2 2023-06-27T12:38:54.632 INFO:tasks.rados.rados.0.smithi138.stdout:283: snap_create 2023-06-27T12:38:54.633 INFO:tasks.rados.rados.0.smithi138.stdout:278: finishing write tid 3 to smithi138138488-37 2023-06-27T12:38:54.633 INFO:tasks.rados.rados.0.smithi138.stdout:278: finishing write tid 4 to smithi138138488-37 2023-06-27T12:38:54.633 INFO:tasks.rados.rados.0.smithi138.stdout:278: oid 37 updating version 0 to 44 2023-06-27T12:38:54.634 INFO:tasks.rados.rados.0.smithi138.stdout:278: oid 37 updating version 44 to 47 2023-06-27T12:38:54.634 INFO:tasks.rados.rados.0.smithi138.stdout:278: oid 37 version 47 is already newer than 47 2023-06-27T12:38:54.634 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 47 (ObjNum 160 snap 24 seq_num 160) dirty exists 2023-06-27T12:38:54.634 INFO:tasks.rados.rados.0.smithi138.stdout:278: left oid 37 (ObjNum 160 snap 24 seq_num 160) 2023-06-27T12:38:54.635 INFO:tasks.rados.rados.0.smithi138.stdout:279: finishing write tid 2 to smithi138138488-50 2023-06-27T12:38:54.635 INFO:tasks.rados.rados.0.smithi138.stdout:279: finishing write tid 3 to smithi138138488-50 2023-06-27T12:38:54.635 INFO:tasks.rados.rados.0.smithi138.stdout:279: finishing write tid 4 to smithi138138488-50 2023-06-27T12:38:54.635 INFO:tasks.rados.rados.0.smithi138.stdout:279: oid 50 updating version 0 to 14 2023-06-27T12:38:54.636 INFO:tasks.rados.rados.0.smithi138.stdout:279: oid 50 version 14 is already newer than 11 2023-06-27T12:38:54.636 INFO:tasks.rados.rados.0.smithi138.stdout:279: oid 50 version 14 is already newer than 14 2023-06-27T12:38:54.636 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 14 (ObjNum 161 snap 24 seq_num 161) dirty exists 2023-06-27T12:38:54.637 INFO:tasks.rados.rados.0.smithi138.stdout:279: left oid 50 (ObjNum 161 snap 24 seq_num 161) 2023-06-27T12:38:54.652 INFO:tasks.rados.rados.0.smithi138.stdout:281: finishing write tid 1 to smithi138138488-6 2023-06-27T12:38:54.703 INFO:tasks.rados.rados.0.smithi138.stdout:281: finishing write tid 2 to smithi138138488-6 2023-06-27T12:38:54.712 INFO:tasks.rados.rados.0.smithi138.stdout:281: finishing write tid 3 to smithi138138488-6 2023-06-27T12:38:54.713 INFO:tasks.rados.rados.0.smithi138.stdout:282: finishing write tid 1 to smithi138138488-7 2023-06-27T12:38:54.713 INFO:tasks.rados.rados.0.smithi138.stdout:281: finishing write tid 4 to smithi138138488-6 2023-06-27T12:38:54.713 INFO:tasks.rados.rados.0.smithi138.stdout:281: oid 6 updating version 0 to 39 2023-06-27T12:38:54.713 INFO:tasks.rados.rados.0.smithi138.stdout:281: oid 6 version 39 is already newer than 39 2023-06-27T12:38:54.713 INFO:tasks.rados.rados.0.smithi138.stdout:281: oid 6 version 39 is already newer than 36 2023-06-27T12:38:54.713 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 39 (ObjNum 162 snap 24 seq_num 162) dirty exists 2023-06-27T12:38:54.714 INFO:tasks.rados.rados.0.smithi138.stdout:281: left oid 6 (ObjNum 162 snap 24 seq_num 162) 2023-06-27T12:38:54.714 INFO:tasks.rados.rados.0.smithi138.stdout:282: finishing write tid 2 to smithi138138488-7 2023-06-27T12:38:54.714 INFO:tasks.rados.rados.0.smithi138.stdout:282: finishing write tid 3 to smithi138138488-7 2023-06-27T12:38:54.716 INFO:tasks.rados.rados.0.smithi138.stdout:282: finishing write tid 4 to smithi138138488-7 2023-06-27T12:38:54.716 INFO:tasks.rados.rados.0.smithi138.stdout:282: oid 7 updating version 0 to 49 2023-06-27T12:38:54.716 INFO:tasks.rados.rados.0.smithi138.stdout:282: oid 7 version 49 is already newer than 49 2023-06-27T12:38:54.716 INFO:tasks.rados.rados.0.smithi138.stdout:282: oid 7 version 49 is already newer than 49 2023-06-27T12:38:54.717 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 49 (ObjNum 163 snap 24 seq_num 163) dirty exists 2023-06-27T12:38:54.717 INFO:tasks.rados.rados.0.smithi138.stdout:282: left oid 7 (ObjNum 163 snap 24 seq_num 163) 2023-06-27T12:38:55.379 INFO:tasks.rados.rados.0.smithi138.stdout:278: done (4 left) 2023-06-27T12:38:55.379 INFO:tasks.rados.rados.0.smithi138.stdout:279: done (3 left) 2023-06-27T12:38:55.379 INFO:tasks.rados.rados.0.smithi138.stdout:281: done (2 left) 2023-06-27T12:38:55.380 INFO:tasks.rados.rados.0.smithi138.stdout:282: done (1 left) 2023-06-27T12:38:55.380 INFO:tasks.rados.rados.0.smithi138.stdout:283: done (0 left) 2023-06-27T12:38:55.380 INFO:tasks.rados.rados.0.smithi138.stdout:284: read oid 13 snap -1 2023-06-27T12:38:55.380 INFO:tasks.rados.rados.0.smithi138.stdout:284: expect (ObjNum 93 snap 7 seq_num 93) 2023-06-27T12:38:55.380 INFO:tasks.rados.rados.0.smithi138.stdout:285: snap_remove snap 24 2023-06-27T12:38:56.031 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:38:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:56.152+0000 7eff58c52700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:56.156+0000 7eff58c52700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:56.359 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:38:56.359 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:38:56.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:38:56.380 INFO:tasks.rados.rados.0.smithi138.stdout:285: done (1 left) 2023-06-27T12:38:56.380 INFO:tasks.rados.rados.0.smithi138.stdout:286: write oid 9 current snap is 25 2023-06-27T12:38:56.380 INFO:tasks.rados.rados.0.smithi138.stdout:286: seq_num 164 ranges {756348=784249,2063988=686387} 2023-06-27T12:38:56.392 INFO:tasks.rados.rados.0.smithi138.stdout:286: writing smithi138138488-9 from 756348 to 1540597 tid 1 2023-06-27T12:38:56.401 INFO:tasks.rados.rados.0.smithi138.stdout:286: writing smithi138138488-9 from 2063988 to 2750375 tid 2 2023-06-27T12:38:56.402 INFO:tasks.rados.rados.0.smithi138.stdout:287: delete oid 26 current snap is 25 2023-06-27T12:38:56.402 INFO:tasks.rados.rados.0.smithi138.stdout:284: expect (ObjNum 93 snap 7 seq_num 93) 2023-06-27T12:38:56.497 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:56.490+0000 7fe39c9da700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:56.500 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:56.496+0000 7fe39c9da700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:56.588 INFO:tasks.rados.rados.0.smithi138.stdout:286: finishing write tid 1 to smithi138138488-9 2023-06-27T12:38:56.589 INFO:tasks.rados.rados.0.smithi138.stdout:284: done (2 left) 2023-06-27T12:38:56.589 INFO:tasks.rados.rados.0.smithi138.stdout:287: done (1 left) 2023-06-27T12:38:56.589 INFO:tasks.rados.rados.0.smithi138.stdout:288: read oid 6 snap 23 2023-06-27T12:38:56.589 INFO:tasks.rados.rados.0.smithi138.stdout:288: expect (ObjNum 66 snap 3 seq_num 66) 2023-06-27T12:38:56.589 INFO:tasks.rados.rados.0.smithi138.stdout:289: snap_create 2023-06-27T12:38:56.590 INFO:tasks.rados.rados.0.smithi138.stdout:286: finishing write tid 2 to smithi138138488-9 2023-06-27T12:38:56.590 INFO:tasks.rados.rados.0.smithi138.stdout:286: finishing write tid 3 to smithi138138488-9 2023-06-27T12:38:56.590 INFO:tasks.rados.rados.0.smithi138.stdout:286: finishing write tid 4 to smithi138138488-9 2023-06-27T12:38:56.590 INFO:tasks.rados.rados.0.smithi138.stdout:286: oid 9 updating version 0 to 73 2023-06-27T12:38:56.590 INFO:tasks.rados.rados.0.smithi138.stdout:286: oid 9 version 73 is already newer than 73 2023-06-27T12:38:56.590 INFO:tasks.rados.rados.0.smithi138.stdout:286: oid 9 version 73 is already newer than 73 2023-06-27T12:38:56.590 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 73 (ObjNum 164 snap 25 seq_num 164) dirty exists 2023-06-27T12:38:56.591 INFO:tasks.rados.rados.0.smithi138.stdout:286: left oid 9 (ObjNum 164 snap 25 seq_num 164) 2023-06-27T12:38:56.619 INFO:tasks.rados.rados.0.smithi138.stdout:288: expect (ObjNum 66 snap 3 seq_num 66) 2023-06-27T12:38:56.718 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:38:56.719 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:38:56.731 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-06-27T12:38:56.375435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":100322611,"num_objects":102,"num_object_clones":52,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":12,"num_read":586,"num_read_kb":296045,"num_write":691,"num_write_kb":160293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":697,"ondisk_log_size":697,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":405960,"kb_used_data":298664,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377081400,"statfs":{"total":386547056640,"available":386131353600,"internally_reserved":0,"allocated":305831936,"data_stored":321188153,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9983226,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":18,"num_read_kb":11230,"num_write":16,"num_write_kb":4558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.787322"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":110,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.932448+0000","last_change":"2023-06-27T12:38:53.936635+0000","last_active":"2023-06-27T12:38:53.932448+0000","last_peered":"2023-06-27T12:38:53.932448+0000","last_clean":"2023-06-27T12:38:53.932448+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:53.932448+0000","last_undegraded":"2023-06-27T12:38:53.932448+0000","last_fullsized":"2023-06-27T12:38:53.932448+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"59'47","reported_seq":178,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.612608+0000","last_change":"2023-06-27T12:38:53.738944+0000","last_active":"2023-06-27T12:38:53.612608+0000","last_peered":"2023-06-27T12:38:53.612608+0000","last_clean":"2023-06-27T12:38:53.612608+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:53.612608+0000","last_undegraded":"2023-06-27T12:38:53.612608+0000","last_fullsized":"2023-06-27T12:38:53.612608+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8537890,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":29,"num_read_kb":20814,"num_write":44,"num_write_kb":12565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"65'47","reported_seq":169,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.887722+0000","last_change":"2023-06-27T12:38:53.840202+0000","last_active":"2023-06-27T12:38:53.887722+0000","last_peered":"2023-06-27T12:38:53.887722+0000","last_clean":"2023-06-27T12:38:53.887722+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:53.887722+0000","last_undegraded":"2023-06-27T12:38:53.887722+0000","last_fullsized":"2023-06-27T12:38:53.887722+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5613505,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":12455,"num_write":47,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.930360+0000","last_change":"2023-06-27T12:38:53.938779+0000","last_active":"2023-06-27T12:38:53.930360+0000","last_peered":"2023-06-27T12:38:53.930360+0000","last_clean":"2023-06-27T12:38:53.930360+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:53.930360+0000","last_undegraded":"2023-06-27T12:38:53.930360+0000","last_fullsized":"2023-06-27T12:38:53.930360+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":8347,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"48'17","reported_seq":135,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.619003+0000","last_change":"2023-06-27T12:38:53.623461+0000","last_active":"2023-06-27T12:38:53.619003+0000","last_peered":"2023-06-27T12:38:53.619003+0000","last_clean":"2023-06-27T12:38:53.619003+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:53.619003+0000","last_undegraded":"2023-06-27T12:38:53.619003+0000","last_fullsized":"2023-06-27T12:38:53.619003+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268978,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":14490,"num_write":21,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"64'58","reported_seq":186,"reported_epoch":65,"state":"active+clean+snaptrim","last_fresh":"2023-06-27T12:38:53.931460+0000","last_change":"2023-06-27T12:38:53.931460+0000","last_active":"2023-06-27T12:38:53.931460+0000","last_peered":"2023-06-27T12:38:53.931460+0000","last_clean":"2023-06-27T12:38:53.931460+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:53.931460+0000","last_undegraded":"2023-06-27T12:38:53.931460+0000","last_fullsized":"2023-06-27T12:38:53.931460+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11748027,"num_objects":11,"num_object_clones":6,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":78,"num_read_kb":47699,"num_write":53,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"64'43","reported_seq":157,"reported_epoch":65,"state":"active+clean+snaptrim","last_fresh":"2023-06-27T12:38:53.932537+0000","last_change":"2023-06-27T12:38:53.932537+0000","last_active":"2023-06-27T12:38:53.932537+0000","last_peered":"2023-06-27T12:38:53.932537+0000","last_clean":"2023-06-27T12:38:53.932537+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:53.932537+0000","last_undegraded":"2023-06-27T12:38:53.932537+0000","last_fullsized":"2023-06-27T12:38:53.932537+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571504,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":34,"num_read_kb":17110,"num_write":38,"num_write_kb":11366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"65'14","reported_seq":135,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.946798+0000","last_change":"2023-06-27T12:38:53.828677+0000","last_active":"2023-06-27T12:38:53.946798+0000","last_peered":"2023-06-27T12:38:53.946798+0000","last_clean":"2023-06-27T12:38:53.946798+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:53.946798+0000","last_undegraded":"2023-06-27T12:38:53.946798+0000","last_fullsized":"2023-06-27T12:38:53.946798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2705927,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":18243,"num_write":13,"num_write_kb":3961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":101,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.632854+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:53.632854+0000","last_peered":"2023-06-27T12:38:53.632854+0000","last_clean":"2023-06-27T12:38:53.632854+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:53.632854+0000","last_undegraded":"2023-06-27T12:38:53.632854+0000","last_fullsized":"2023-06-27T12:38:53.632854+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":111,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.832373+0000","last_change":"2023-06-27T12:38:53.840381+0000","last_active":"2023-06-27T12:38:53.832373+0000","last_peered":"2023-06-27T12:38:53.832373+0000","last_clean":"2023-06-27T12:38:53.832373+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:53.832373+0000","last_undegraded":"2023-06-27T12:38:53.832373+0000","last_fullsized":"2023-06-27T12:38:53.832373+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":87,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.926223+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:53.926223+0000","last_peered":"2023-06-27T12:38:53.926223+0000","last_clean":"2023-06-27T12:38:53.926223+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:53.926223+0000","last_undegraded":"2023-06-27T12:38:53.926223+0000","last_fullsized":"2023-06-27T12:38:53.926223+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"47'34","reported_seq":159,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.637124+0000","last_change":"2023-06-27T12:38:53.653879+0000","last_active":"2023-06-27T12:38:53.637124+0000","last_peered":"2023-06-27T12:38:53.637124+0000","last_clean":"2023-06-27T12:38:53.637124+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:53.637124+0000","last_undegraded":"2023-06-27T12:38:53.637124+0000","last_fullsized":"2023-06-27T12:38:53.637124+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294172,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":19,"num_read_kb":10007,"num_write":29,"num_write_kb":7882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":93,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.824226+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:53.824226+0000","last_peered":"2023-06-27T12:38:53.824226+0000","last_clean":"2023-06-27T12:38:53.824226+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:53.824226+0000","last_undegraded":"2023-06-27T12:38:53.824226+0000","last_fullsized":"2023-06-27T12:38:53.824226+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'92","reported_seq":211,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.635513+0000","last_change":"2023-06-27T12:38:53.699550+0000","last_active":"2023-06-27T12:38:53.635513+0000","last_peered":"2023-06-27T12:38:53.635513+0000","last_clean":"2023-06-27T12:38:53.635513+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:53.635513+0000","last_undegraded":"2023-06-27T12:38:53.635513+0000","last_fullsized":"2023-06-27T12:38:53.635513+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11832062,"num_objects":12,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":54,"num_read_kb":43540,"num_write":83,"num_write_kb":24385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":93,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.828132+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:53.828132+0000","last_peered":"2023-06-27T12:38:53.828132+0000","last_clean":"2023-06-27T12:38:53.828132+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:53.828132+0000","last_undegraded":"2023-06-27T12:38:53.828132+0000","last_fullsized":"2023-06-27T12:38:53.828132+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":218,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.609752+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:53.609752+0000","last_peered":"2023-06-27T12:38:53.609752+0000","last_clean":"2023-06-27T12:38:53.609752+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:53.609752+0000","last_undegraded":"2023-06-27T12:38:53.609752+0000","last_fullsized":"2023-06-27T12:38:53.609752+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"62'67","reported_seq":200,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.832798+0000","last_change":"2023-06-27T12:38:53.870148+0000","last_active":"2023-06-27T12:38:53.832798+0000","last_peered":"2023-06-27T12:38:53.832798+0000","last_clean":"2023-06-27T12:38:53.832798+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:53.832798+0000","last_undegraded":"2023-06-27T12:38:53.832798+0000","last_fullsized":"2023-06-27T12:38:53.832798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13176622,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":70,"num_read_kb":27018,"num_write":65,"num_write_kb":17962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.926289+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:53.926289+0000","last_peered":"2023-06-27T12:38:53.926289+0000","last_clean":"2023-06-27T12:38:53.926289+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:53.926289+0000","last_undegraded":"2023-06-27T12:38:53.926289+0000","last_fullsized":"2023-06-27T12:38:53.926289+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'32","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.616348+0000","last_change":"2023-06-27T12:38:53.683981+0000","last_active":"2023-06-27T12:38:53.616348+0000","last_peered":"2023-06-27T12:38:53.616348+0000","last_clean":"2023-06-27T12:38:53.616348+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:53.616348+0000","last_undegraded":"2023-06-27T12:38:53.616348+0000","last_fullsized":"2023-06-27T12:38:53.616348+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3966419,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":87,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.610035+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:53.610035+0000","last_peered":"2023-06-27T12:38:53.610035+0000","last_clean":"2023-06-27T12:38:53.610035+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:53.610035+0000","last_undegraded":"2023-06-27T12:38:53.610035+0000","last_fullsized":"2023-06-27T12:38:53.610035+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"57'44","reported_seq":171,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.618403+0000","last_change":"2023-06-27T12:38:53.639193+0000","last_active":"2023-06-27T12:38:53.618403+0000","last_peered":"2023-06-27T12:38:53.618403+0000","last_clean":"2023-06-27T12:38:53.618403+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:53.618403+0000","last_undegraded":"2023-06-27T12:38:53.618403+0000","last_fullsized":"2023-06-27T12:38:53.618403+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11126135,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":27,"num_read_kb":21411,"num_write":41,"num_write_kb":12814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"57'24","reported_seq":133,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.617392+0000","last_change":"2023-06-27T12:38:53.632223+0000","last_active":"2023-06-27T12:38:53.617392+0000","last_peered":"2023-06-27T12:38:53.617392+0000","last_clean":"2023-06-27T12:38:53.617392+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:53.617392+0000","last_undegraded":"2023-06-27T12:38:53.617392+0000","last_fullsized":"2023-06-27T12:38:53.617392+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4806802,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":13726,"num_write":21,"num_write_kb":5863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.612604+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:53.612604+0000","last_peered":"2023-06-27T12:38:53.612604+0000","last_clean":"2023-06-27T12:38:53.612604+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:53.612604+0000","last_undegraded":"2023-06-27T12:38:53.612604+0000","last_fullsized":"2023-06-27T12:38:53.612604+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":93,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.821415+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:53.821415+0000","last_peered":"2023-06-27T12:38:53.821415+0000","last_clean":"2023-06-27T12:38:53.821415+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:53.821415+0000","last_undegraded":"2023-06-27T12:38:53.821415+0000","last_fullsized":"2023-06-27T12:38:53.821415+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"55'41","reported_seq":169,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-06-27T12:38:53.626527+0000","last_change":"2023-06-27T12:38:53.629639+0000","last_active":"2023-06-27T12:38:53.626527+0000","last_peered":"2023-06-27T12:38:53.626527+0000","last_clean":"2023-06-27T12:38:53.626527+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:53.626527+0000","last_undegraded":"2023-06-27T12:38:53.626527+0000","last_fullsized":"2023-06-27T12:38:53.626527+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7949678,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":22468,"num_write":35,"num_write_kb":10153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":99732243,"num_objects":100,"num_object_clones":52,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":12,"num_read":483,"num_read_kb":295954,"num_write":554,"num_write_kb":158399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105236,"kb_used_data":78412,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94266604,"statfs":{"total":96636764160,"available":96529002496,"internally_reserved":0,"allocated":80293888,"data_stored":83788527,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100320,"kb_used_data":73496,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94271520,"statfs":{"total":96636764160,"available":96534036480,"internally_reserved":0,"allocated":75259904,"data_stored":78427376,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705725,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96232,"kb_used_data":69408,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94275608,"statfs":{"total":96636764160,"available":96538222592,"internally_reserved":0,"allocated":71073792,"data_stored":75447926,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":104172,"kb_used_data":77348,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94267668,"statfs":{"total":96636764160,"available":96530092032,"internally_reserved":0,"allocated":79204352,"data_stored":83524324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:38:57.379 INFO:tasks.rados.rados.0.smithi138.stdout:286: done (2 left) 2023-06-27T12:38:57.379 INFO:tasks.rados.rados.0.smithi138.stdout:288: done (1 left) 2023-06-27T12:38:57.379 INFO:tasks.rados.rados.0.smithi138.stdout:289: done (0 left) 2023-06-27T12:38:57.380 INFO:tasks.rados.rados.0.smithi138.stdout:290: write (excl) oid 7 current snap is 26 2023-06-27T12:38:57.380 INFO:tasks.rados.rados.0.smithi138.stdout:290: seq_num 166 ranges {789962=534747,1780952=690056} 2023-06-27T12:38:57.387 INFO:tasks.rados.rados.0.smithi138.stdout:290: writing smithi138138488-7 from 789962 to 1324709 tid 1 2023-06-27T12:38:57.395 INFO:tasks.rados.rados.0.smithi138.stdout:290: writing smithi138138488-7 from 1780952 to 2471008 tid 2 2023-06-27T12:38:57.396 INFO:tasks.rados.rados.0.smithi138.stdout:291: read oid 50 snap -1 2023-06-27T12:38:57.396 INFO:tasks.rados.rados.0.smithi138.stdout:291: expect (ObjNum 161 snap 24 seq_num 161) 2023-06-27T12:38:57.396 INFO:tasks.rados.rados.0.smithi138.stdout:292: read oid 36 snap -1 2023-06-27T12:38:57.396 INFO:tasks.rados.rados.0.smithi138.stdout:292: expect deleted 2023-06-27T12:38:57.396 INFO:tasks.rados.rados.0.smithi138.stdout:293: read oid 20 snap 23 2023-06-27T12:38:57.396 INFO:tasks.rados.rados.0.smithi138.stdout:293: expect (ObjNum 153 snap 21 seq_num 153) 2023-06-27T12:38:57.396 INFO:tasks.rados.rados.0.smithi138.stdout:294: delete oid 23 current snap is 26 2023-06-27T12:38:57.414 INFO:tasks.rados.rados.0.smithi138.stdout:291: expect (ObjNum 161 snap 24 seq_num 161) 2023-06-27T12:38:57.588 INFO:tasks.rados.rados.0.smithi138.stdout:293: expect (ObjNum 153 snap 21 seq_num 153) 2023-06-27T12:38:57.717 INFO:tasks.rados.rados.0.smithi138.stdout:290: finishing write tid 1 to smithi138138488-7 2023-06-27T12:38:57.736 INFO:tasks.rados.rados.0.smithi138.stdout:290: finishing write tid 2 to smithi138138488-7 2023-06-27T12:38:57.737 INFO:tasks.rados.rados.0.smithi138.stdout:290: finishing write tid 3 to smithi138138488-7 2023-06-27T12:38:57.737 INFO:tasks.rados.rados.0.smithi138.stdout:290: finishing write tid 4 to smithi138138488-7 2023-06-27T12:38:57.737 INFO:tasks.rados.rados.0.smithi138.stdout:290: oid 7 updating version 0 to 52 2023-06-27T12:38:57.737 INFO:tasks.rados.rados.0.smithi138.stdout:290: oid 7 updating version 52 to 55 2023-06-27T12:38:57.737 INFO:tasks.rados.rados.0.smithi138.stdout:290: oid 7 version 55 is already newer than 55 2023-06-27T12:38:57.738 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 55 (ObjNum 166 snap 26 seq_num 166) dirty exists 2023-06-27T12:38:57.738 INFO:tasks.rados.rados.0.smithi138.stdout:290: left oid 7 (ObjNum 166 snap 26 seq_num 166) 2023-06-27T12:38:57.738 INFO:tasks.rados.rados.0.smithi138.stdout:290: done (4 left) 2023-06-27T12:38:57.738 INFO:tasks.rados.rados.0.smithi138.stdout:291: done (3 left) 2023-06-27T12:38:57.738 INFO:tasks.rados.rados.0.smithi138.stdout:292: done (2 left) 2023-06-27T12:38:57.738 INFO:tasks.rados.rados.0.smithi138.stdout:293: done (1 left) 2023-06-27T12:38:57.738 INFO:tasks.rados.rados.0.smithi138.stdout:294: done (0 left) 2023-06-27T12:38:57.738 INFO:tasks.rados.rados.0.smithi138.stdout:295: write oid 29 current snap is 26 2023-06-27T12:38:57.739 INFO:tasks.rados.rados.0.smithi138.stdout:295: seq_num 168 ranges {423576=401598,1497916=693725} 2023-06-27T12:38:57.739 INFO:tasks.rados.rados.0.smithi138.stdout:295: writing smithi138138488-29 from 423576 to 825174 tid 1 2023-06-27T12:38:57.740 INFO:tasks.rados.rados.0.smithi138.stdout:295: writing smithi138138488-29 from 1497916 to 2191641 tid 2 2023-06-27T12:38:57.740 INFO:tasks.rados.rados.0.smithi138.stdout:296: write oid 28 current snap is 26 2023-06-27T12:38:57.740 INFO:tasks.rados.rados.0.smithi138.stdout:296: seq_num 169 ranges {440383=476847,1556398=437383,2474721=365662} 2023-06-27T12:38:57.740 INFO:tasks.rados.rados.0.smithi138.stdout:296: writing smithi138138488-28 from 440383 to 917230 tid 1 2023-06-27T12:38:57.740 INFO:tasks.rados.rados.0.smithi138.stdout:296: writing smithi138138488-28 from 1556398 to 1993781 tid 2 2023-06-27T12:38:57.745 INFO:tasks.rados.rados.0.smithi138.stdout:296: writing smithi138138488-28 from 2474721 to 2840383 tid 3 2023-06-27T12:38:57.745 INFO:tasks.rados.rados.0.smithi138.stdout:297: snap_create 2023-06-27T12:38:57.767 INFO:tasks.rados.rados.0.smithi138.stdout:296: finishing write tid 1 to smithi138138488-28 2023-06-27T12:38:57.777 INFO:tasks.rados.rados.0.smithi138.stdout:296: finishing write tid 2 to smithi138138488-28 2023-06-27T12:38:57.790 INFO:tasks.rados.rados.0.smithi138.stdout:296: finishing write tid 3 to smithi138138488-28 2023-06-27T12:38:57.805 INFO:tasks.rados.rados.0.smithi138.stdout:296: finishing write tid 4 to smithi138138488-28 2023-06-27T12:38:57.809 INFO:tasks.rados.rados.0.smithi138.stdout:296: finishing write tid 5 to smithi138138488-28 2023-06-27T12:38:57.809 INFO:tasks.rados.rados.0.smithi138.stdout:296: oid 28 updating version 0 to 59 2023-06-27T12:38:57.809 INFO:tasks.rados.rados.0.smithi138.stdout:296: oid 28 version 59 is already newer than 59 2023-06-27T12:38:57.809 INFO:tasks.rados.rados.0.smithi138.stdout:296: oid 28 version 59 is already newer than 59 2023-06-27T12:38:57.809 INFO:tasks.rados.rados.0.smithi138.stdout:296: oid 28 version 59 is already newer than 57 2023-06-27T12:38:57.810 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 59 (ObjNum 169 snap 26 seq_num 169) dirty exists 2023-06-27T12:38:57.810 INFO:tasks.rados.rados.0.smithi138.stdout:296: left oid 28 (ObjNum 169 snap 26 seq_num 169) 2023-06-27T12:38:57.835 INFO:tasks.rados.rados.0.smithi138.stdout:295: finishing write tid 1 to smithi138138488-29 2023-06-27T12:38:57.835 INFO:tasks.rados.rados.0.smithi138.stdout:295: finishing write tid 2 to smithi138138488-29 2023-06-27T12:38:57.836 INFO:tasks.rados.rados.0.smithi138.stdout:295: finishing write tid 3 to smithi138138488-29 2023-06-27T12:38:57.837 INFO:tasks.rados.rados.0.smithi138.stdout:295: finishing write tid 4 to smithi138138488-29 2023-06-27T12:38:57.837 INFO:tasks.rados.rados.0.smithi138.stdout:295: oid 29 updating version 0 to 28 2023-06-27T12:38:57.837 INFO:tasks.rados.rados.0.smithi138.stdout:295: oid 29 version 28 is already newer than 28 2023-06-27T12:38:57.837 INFO:tasks.rados.rados.0.smithi138.stdout:295: oid 29 version 28 is already newer than 28 2023-06-27T12:38:57.837 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 28 (ObjNum 168 snap 26 seq_num 168) dirty exists 2023-06-27T12:38:57.838 INFO:tasks.rados.rados.0.smithi138.stdout:295: left oid 29 (ObjNum 168 snap 26 seq_num 168) 2023-06-27T12:38:58.230 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:38:58.356 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:58.351+0000 7f3f2a414700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:58.359 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:38:58.355+0000 7f3f2a414700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:38:58.384 INFO:tasks.rados.rados.0.smithi138.stdout:295: done (2 left) 2023-06-27T12:38:58.384 INFO:tasks.rados.rados.0.smithi138.stdout:296: done (1 left) 2023-06-27T12:38:58.384 INFO:tasks.rados.rados.0.smithi138.stdout:297: done (0 left) 2023-06-27T12:38:58.385 INFO:tasks.rados.rados.0.smithi138.stdout:298: read oid 24 snap 25 2023-06-27T12:38:58.385 INFO:tasks.rados.rados.0.smithi138.stdout:298: expect (ObjNum 73 snap 3 seq_num 73) 2023-06-27T12:38:58.385 INFO:tasks.rados.rados.0.smithi138.stdout:299: snap_remove snap 23 2023-06-27T12:38:59.386 INFO:tasks.rados.rados.0.smithi138.stdout:299: done (1 left) 2023-06-27T12:38:59.386 INFO:tasks.rados.rados.0.smithi138.stdout:300: read oid 6 snap -1 2023-06-27T12:38:59.387 INFO:tasks.rados.rados.0.smithi138.stdout:300: expect (ObjNum 162 snap 24 seq_num 162) 2023-06-27T12:38:59.387 INFO:tasks.rados.rados.0.smithi138.stdout:301: snap_remove snap 26 2023-06-27T12:39:00.379 INFO:tasks.rados.rados.0.smithi138.stdout:301: done (2 left) 2023-06-27T12:39:00.379 INFO:tasks.rados.rados.0.smithi138.stdout:302: write oid 31 current snap is 27 2023-06-27T12:39:00.379 INFO:tasks.rados.rados.0.smithi138.stdout:302: seq_num 170 ranges {457190=552096,1614880=581041,2785791=71399} 2023-06-27T12:39:00.384 INFO:tasks.rados.rados.0.smithi138.stdout:302: writing smithi138138488-31 from 457190 to 1009286 tid 1 2023-06-27T12:39:00.391 INFO:tasks.rados.rados.0.smithi138.stdout:302: writing smithi138138488-31 from 1614880 to 2195921 tid 2 2023-06-27T12:39:00.395 INFO:tasks.rados.rados.0.smithi138.stdout:302: writing smithi138138488-31 from 2785791 to 2857190 tid 3 2023-06-27T12:39:00.395 INFO:tasks.rados.rados.0.smithi138.stdout:303: read oid 39 snap -1 2023-06-27T12:39:00.396 INFO:tasks.rados.rados.0.smithi138.stdout:303: expect deleted 2023-06-27T12:39:00.396 INFO:tasks.rados.rados.0.smithi138.stdout:304: write (excl) oid 17 current snap is 27 2023-06-27T12:39:00.396 INFO:tasks.rados.rados.0.smithi138.stdout:304: seq_num 171 ranges {473997=627345,1673362=441052,2529567=344430} 2023-06-27T12:39:00.404 INFO:tasks.rados.rados.0.smithi138.stdout:304: writing smithi138138488-17 from 473997 to 1101342 tid 1 2023-06-27T12:39:00.411 INFO:tasks.rados.rados.0.smithi138.stdout:304: writing smithi138138488-17 from 1673362 to 2114414 tid 2 2023-06-27T12:39:00.417 INFO:tasks.rados.rados.0.smithi138.stdout:304: writing smithi138138488-17 from 2529567 to 2873997 tid 3 2023-06-27T12:39:00.417 INFO:tasks.rados.rados.0.smithi138.stdout:305: snap_create 2023-06-27T12:39:00.417 INFO:tasks.rados.rados.0.smithi138.stdout:298: expect (ObjNum 73 snap 3 seq_num 73) 2023-06-27T12:39:00.439 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:00.589 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:00.584+0000 7fde4cb55700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:00.594 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:00.590+0000 7fde4cb55700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:00.646 INFO:tasks.rados.rados.0.smithi138.stdout:300: expect (ObjNum 162 snap 24 seq_num 162) 2023-06-27T12:39:00.801 INFO:tasks.rados.rados.0.smithi138.stdout:304: finishing write tid 1 to smithi138138488-17 2023-06-27T12:39:00.801 INFO:tasks.rados.rados.0.smithi138.stdout:302: finishing write tid 1 to smithi138138488-31 2023-06-27T12:39:00.802 INFO:tasks.rados.rados.0.smithi138.stdout:302: finishing write tid 2 to smithi138138488-31 2023-06-27T12:39:00.802 INFO:tasks.rados.rados.0.smithi138.stdout:302: finishing write tid 3 to smithi138138488-31 2023-06-27T12:39:00.802 INFO:tasks.rados.rados.0.smithi138.stdout:304: finishing write tid 2 to smithi138138488-17 2023-06-27T12:39:00.802 INFO:tasks.rados.rados.0.smithi138.stdout:302: finishing write tid 4 to smithi138138488-31 2023-06-27T12:39:00.802 INFO:tasks.rados.rados.0.smithi138.stdout:302: finishing write tid 5 to smithi138138488-31 2023-06-27T12:39:00.802 INFO:tasks.rados.rados.0.smithi138.stdout:302: oid 31 updating version 0 to 69 2023-06-27T12:39:00.802 INFO:tasks.rados.rados.0.smithi138.stdout:302: oid 31 version 69 is already newer than 69 2023-06-27T12:39:00.802 INFO:tasks.rados.rados.0.smithi138.stdout:302: oid 31 version 69 is already newer than 69 2023-06-27T12:39:00.803 INFO:tasks.rados.rados.0.smithi138.stdout:302: oid 31 version 69 is already newer than 69 2023-06-27T12:39:00.803 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 69 (ObjNum 170 snap 27 seq_num 170) dirty exists 2023-06-27T12:39:00.803 INFO:tasks.rados.rados.0.smithi138.stdout:302: left oid 31 (ObjNum 170 snap 27 seq_num 170) 2023-06-27T12:39:00.803 INFO:tasks.rados.rados.0.smithi138.stdout:304: finishing write tid 3 to smithi138138488-17 2023-06-27T12:39:00.803 INFO:tasks.rados.rados.0.smithi138.stdout:304: finishing write tid 4 to smithi138138488-17 2023-06-27T12:39:00.803 INFO:tasks.rados.rados.0.smithi138.stdout:304: finishing write tid 5 to smithi138138488-17 2023-06-27T12:39:00.803 INFO:tasks.rados.rados.0.smithi138.stdout:304: oid 17 updating version 0 to 53 2023-06-27T12:39:00.804 INFO:tasks.rados.rados.0.smithi138.stdout:304: oid 17 version 53 is already newer than 53 2023-06-27T12:39:00.804 INFO:tasks.rados.rados.0.smithi138.stdout:304: oid 17 version 53 is already newer than 53 2023-06-27T12:39:00.804 INFO:tasks.rados.rados.0.smithi138.stdout:304: oid 17 version 53 is already newer than 50 2023-06-27T12:39:00.804 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 53 (ObjNum 171 snap 27 seq_num 171) dirty exists 2023-06-27T12:39:00.804 INFO:tasks.rados.rados.0.smithi138.stdout:304: left oid 17 (ObjNum 171 snap 27 seq_num 171) 2023-06-27T12:39:01.397 INFO:tasks.rados.rados.0.smithi138.stdout:298: done (5 left) 2023-06-27T12:39:01.398 INFO:tasks.rados.rados.0.smithi138.stdout:300: done (4 left) 2023-06-27T12:39:01.398 INFO:tasks.rados.rados.0.smithi138.stdout:302: done (3 left) 2023-06-27T12:39:01.398 INFO:tasks.rados.rados.0.smithi138.stdout:303: done (2 left) 2023-06-27T12:39:01.398 INFO:tasks.rados.rados.0.smithi138.stdout:304: done (1 left) 2023-06-27T12:39:01.398 INFO:tasks.rados.rados.0.smithi138.stdout:305: done (0 left) 2023-06-27T12:39:01.398 INFO:tasks.rados.rados.0.smithi138.stdout:306: read oid 16 snap -1 2023-06-27T12:39:01.398 INFO:tasks.rados.rados.0.smithi138.stdout:306: expect deleted 2023-06-27T12:39:01.399 INFO:tasks.rados.rados.0.smithi138.stdout:307: delete oid 37 current snap is 28 2023-06-27T12:39:01.450 INFO:tasks.rados.rados.0.smithi138.stdout:306: done (1 left) 2023-06-27T12:39:01.450 INFO:tasks.rados.rados.0.smithi138.stdout:307: done (0 left) 2023-06-27T12:39:01.450 INFO:tasks.rados.rados.0.smithi138.stdout:308: snap_remove snap 27 2023-06-27T12:39:01.734 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:01.734 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:01.735 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:01.865 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:01.858+0000 7f6b5bf36700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:01.868 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:01.865+0000 7f6b5bf36700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:02.072 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:02.084 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2023-06-27T12:39:01.917432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98036925,"num_objects":109,"num_object_clones":59,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":13,"num_read":615,"num_read_kb":323593,"num_write":716,"num_write_kb":167623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":732,"ondisk_log_size":732,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":433632,"kb_used_data":326336,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377053728,"statfs":{"total":386547056640,"available":386103017472,"internally_reserved":0,"allocated":334168064,"data_stored":360367497,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2285686,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":29,"num_read_kb":27548,"num_write":25,"num_write_kb":7330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000710"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":122,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.410103+0000","last_change":"2023-06-27T12:38:56.606293+0000","last_active":"2023-06-27T12:38:58.410103+0000","last_peered":"2023-06-27T12:38:58.410103+0000","last_clean":"2023-06-27T12:38:58.410103+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:38:58.410103+0000","last_undegraded":"2023-06-27T12:38:58.410103+0000","last_fullsized":"2023-06-27T12:38:58.410103+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"59'47","reported_seq":191,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.831600+0000","last_change":"2023-06-27T12:38:56.448486+0000","last_active":"2023-06-27T12:38:58.831600+0000","last_peered":"2023-06-27T12:38:58.831600+0000","last_clean":"2023-06-27T12:38:58.831600+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:38:58.831600+0000","last_undegraded":"2023-06-27T12:38:58.831600+0000","last_fullsized":"2023-06-27T12:38:58.831600+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8537890,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":31,"num_read_kb":24619,"num_write":44,"num_write_kb":12565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"65'47","reported_seq":179,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.403648+0000","last_change":"2023-06-27T12:38:56.393015+0000","last_active":"2023-06-27T12:38:58.403648+0000","last_peered":"2023-06-27T12:38:58.403648+0000","last_clean":"2023-06-27T12:38:58.403648+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:38:58.403648+0000","last_undegraded":"2023-06-27T12:38:58.403648+0000","last_fullsized":"2023-06-27T12:38:58.403648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5613505,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":12455,"num_write":47,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":160,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.409139+0000","last_change":"2023-06-27T12:38:56.617112+0000","last_active":"2023-06-27T12:38:58.409139+0000","last_peered":"2023-06-27T12:38:58.409139+0000","last_clean":"2023-06-27T12:38:58.409139+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:38:58.409139+0000","last_undegraded":"2023-06-27T12:38:58.409139+0000","last_fullsized":"2023-06-27T12:38:58.409139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":8347,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"48'17","reported_seq":143,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.808382+0000","last_change":"2023-06-27T12:38:57.768524+0000","last_active":"2023-06-27T12:38:58.808382+0000","last_peered":"2023-06-27T12:38:58.808382+0000","last_clean":"2023-06-27T12:38:58.808382+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:38:58.808382+0000","last_undegraded":"2023-06-27T12:38:58.808382+0000","last_fullsized":"2023-06-27T12:38:58.808382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268978,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":14490,"num_write":21,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"64'58","reported_seq":202,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.421979+0000","last_change":"2023-06-27T12:38:56.604706+0000","last_active":"2023-06-27T12:38:58.421979+0000","last_peered":"2023-06-27T12:38:58.421979+0000","last_clean":"2023-06-27T12:38:58.421979+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:38:58.421979+0000","last_undegraded":"2023-06-27T12:38:58.421979+0000","last_fullsized":"2023-06-27T12:38:58.421979+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11748027,"num_objects":11,"num_object_clones":6,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":85,"num_read_kb":54115,"num_write":53,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"69'59","reported_seq":184,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.411968+0000","last_change":"2023-06-27T12:38:56.617704+0000","last_active":"2023-06-27T12:38:58.411968+0000","last_peered":"2023-06-27T12:38:58.411968+0000","last_clean":"2023-06-27T12:38:58.411968+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:38:58.411968+0000","last_undegraded":"2023-06-27T12:38:58.411968+0000","last_fullsized":"2023-06-27T12:38:58.411968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8084167,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":17113,"num_write":51,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"65'14","reported_seq":148,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.392841+0000","last_change":"2023-06-27T12:38:56.392058+0000","last_active":"2023-06-27T12:38:58.392841+0000","last_peered":"2023-06-27T12:38:58.392841+0000","last_clean":"2023-06-27T12:38:58.392841+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:38:58.392841+0000","last_undegraded":"2023-06-27T12:38:58.392841+0000","last_fullsized":"2023-06-27T12:38:58.392841+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2705927,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":25158,"num_write":13,"num_write_kb":3961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.832096+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:38:58.832096+0000","last_peered":"2023-06-27T12:38:58.832096+0000","last_clean":"2023-06-27T12:38:58.832096+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:38:58.832096+0000","last_undegraded":"2023-06-27T12:38:58.832096+0000","last_fullsized":"2023-06-27T12:38:58.832096+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":121,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.402084+0000","last_change":"2023-06-27T12:38:56.392864+0000","last_active":"2023-06-27T12:38:58.402084+0000","last_peered":"2023-06-27T12:38:58.402084+0000","last_clean":"2023-06-27T12:38:58.402084+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:38:58.402084+0000","last_undegraded":"2023-06-27T12:38:58.402084+0000","last_fullsized":"2023-06-27T12:38:58.402084+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":97,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.406015+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:38:58.406015+0000","last_peered":"2023-06-27T12:38:58.406015+0000","last_clean":"2023-06-27T12:38:58.406015+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:38:58.406015+0000","last_undegraded":"2023-06-27T12:38:58.406015+0000","last_fullsized":"2023-06-27T12:38:58.406015+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"69'41","reported_seq":176,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.822839+0000","last_change":"2023-06-27T12:38:56.448812+0000","last_active":"2023-06-27T12:38:58.822839+0000","last_peered":"2023-06-27T12:38:58.822839+0000","last_clean":"2023-06-27T12:38:58.822839+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:38:58.822839+0000","last_undegraded":"2023-06-27T12:38:58.822839+0000","last_fullsized":"2023-06-27T12:38:58.822839+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722734,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":24,"num_read_kb":16082,"num_write":33,"num_write_kb":9145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.393971+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:38:58.393971+0000","last_peered":"2023-06-27T12:38:58.393971+0000","last_clean":"2023-06-27T12:38:58.393971+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:38:58.393971+0000","last_undegraded":"2023-06-27T12:38:58.393971+0000","last_fullsized":"2023-06-27T12:38:58.393971+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'92","reported_seq":222,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.827529+0000","last_change":"2023-06-27T12:38:57.746533+0000","last_active":"2023-06-27T12:38:58.827529+0000","last_peered":"2023-06-27T12:38:58.827529+0000","last_clean":"2023-06-27T12:38:58.827529+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:38:58.827529+0000","last_undegraded":"2023-06-27T12:38:58.827529+0000","last_fullsized":"2023-06-27T12:38:58.827529+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11832062,"num_objects":12,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":56,"num_read_kb":47872,"num_write":83,"num_write_kb":24385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":101,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.397442+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:38:58.397442+0000","last_peered":"2023-06-27T12:38:58.397442+0000","last_clean":"2023-06-27T12:38:58.397442+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:38:58.397442+0000","last_undegraded":"2023-06-27T12:38:58.397442+0000","last_fullsized":"2023-06-27T12:38:58.397442+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":224,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.820558+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:38:58.820558+0000","last_peered":"2023-06-27T12:38:58.820558+0000","last_clean":"2023-06-27T12:38:58.820558+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:38:58.820558+0000","last_undegraded":"2023-06-27T12:38:58.820558+0000","last_fullsized":"2023-06-27T12:38:58.820558+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"68'73","reported_seq":215,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.402123+0000","last_change":"2023-06-27T12:38:56.396861+0000","last_active":"2023-06-27T12:38:58.402123+0000","last_peered":"2023-06-27T12:38:58.402123+0000","last_clean":"2023-06-27T12:38:58.402123+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:38:58.402123+0000","last_undegraded":"2023-06-27T12:38:58.402123+0000","last_fullsized":"2023-06-27T12:38:58.402123+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13344692,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":71,"num_read_kb":27019,"num_write":69,"num_write_kb":19399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.409737+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:38:58.409737+0000","last_peered":"2023-06-27T12:38:58.409737+0000","last_clean":"2023-06-27T12:38:58.409737+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:38:58.409737+0000","last_undegraded":"2023-06-27T12:38:58.409737+0000","last_fullsized":"2023-06-27T12:38:58.409737+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'32","reported_seq":156,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.827640+0000","last_change":"2023-06-27T12:38:57.753773+0000","last_active":"2023-06-27T12:38:58.827640+0000","last_peered":"2023-06-27T12:38:58.827640+0000","last_clean":"2023-06-27T12:38:58.827640+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:38:58.827640+0000","last_undegraded":"2023-06-27T12:38:58.827640+0000","last_fullsized":"2023-06-27T12:38:58.827640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3966419,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":93,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.830045+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:38:58.830045+0000","last_peered":"2023-06-27T12:38:58.830045+0000","last_clean":"2023-06-27T12:38:58.830045+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:38:58.830045+0000","last_undegraded":"2023-06-27T12:38:58.830045+0000","last_fullsized":"2023-06-27T12:38:58.830045+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"68'46","reported_seq":183,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.822794+0000","last_change":"2023-06-27T12:38:56.439974+0000","last_active":"2023-06-27T12:38:58.822794+0000","last_peered":"2023-06-27T12:38:58.822794+0000","last_clean":"2023-06-27T12:38:58.822794+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:38:58.822794+0000","last_undegraded":"2023-06-27T12:38:58.822794+0000","last_fullsized":"2023-06-27T12:38:58.822794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7428628,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":27,"num_read_kb":21411,"num_write":41,"num_write_kb":12814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"69'28","reported_seq":146,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.809382+0000","last_change":"2023-06-27T12:38:57.746041+0000","last_active":"2023-06-27T12:38:58.809382+0000","last_peered":"2023-06-27T12:38:58.809382+0000","last_clean":"2023-06-27T12:38:58.809382+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:38:58.809382+0000","last_undegraded":"2023-06-27T12:38:58.809382+0000","last_fullsized":"2023-06-27T12:38:58.809382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5109328,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":13727,"num_write":25,"num_write_kb":6934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.825004+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:38:58.825004+0000","last_peered":"2023-06-27T12:38:58.825004+0000","last_clean":"2023-06-27T12:38:58.825004+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:38:58.825004+0000","last_undegraded":"2023-06-27T12:38:58.825004+0000","last_fullsized":"2023-06-27T12:38:58.825004+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.405329+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:38:58.405329+0000","last_peered":"2023-06-27T12:38:58.405329+0000","last_clean":"2023-06-27T12:38:58.405329+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:38:58.405329+0000","last_undegraded":"2023-06-27T12:38:58.405329+0000","last_fullsized":"2023-06-27T12:38:58.405329+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"55'41","reported_seq":179,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-06-27T12:38:58.828777+0000","last_change":"2023-06-27T12:38:56.433134+0000","last_active":"2023-06-27T12:38:58.828777+0000","last_peered":"2023-06-27T12:38:58.828777+0000","last_clean":"2023-06-27T12:38:58.828777+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:38:58.828777+0000","last_undegraded":"2023-06-27T12:38:58.828777+0000","last_fullsized":"2023-06-27T12:38:58.828777+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7949678,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":22468,"num_write":35,"num_write_kb":10153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":97446557,"num_objects":107,"num_object_clones":59,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":13,"num_read":512,"num_read_kb":323502,"num_write":579,"num_write_kb":165729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":635,"ondisk_log_size":635,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673025,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110868,"kb_used_data":84044,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94260972,"statfs":{"total":96636764160,"available":96523235328,"internally_reserved":0,"allocated":86061056,"data_stored":91700944,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673025,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":106160,"kb_used_data":79336,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94265680,"statfs":{"total":96636764160,"available":96528056320,"internally_reserved":0,"allocated":81240064,"data_stored":86884594,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705730,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":104488,"kb_used_data":77664,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94267352,"statfs":{"total":96636764160,"available":96529768448,"internally_reserved":0,"allocated":79527936,"data_stored":86836881,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673025,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":112116,"kb_used_data":85292,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94259724,"statfs":{"total":96636764160,"available":96521957376,"internally_reserved":0,"allocated":87339008,"data_stored":94945078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:02.401 INFO:tasks.rados.rados.0.smithi138.stdout:308: done (0 left) 2023-06-27T12:39:02.401 INFO:tasks.rados.rados.0.smithi138.stdout:309: snap_create 2023-06-27T12:39:02.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:02.845 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:02.840+0000 7fb908ba7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:02.848 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:02.845+0000 7fb908ba7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:03.405 INFO:tasks.rados.rados.0.smithi138.stdout:309: done (0 left) 2023-06-27T12:39:03.405 INFO:tasks.rados.rados.0.smithi138.stdout:310: write oid 32 current snap is 29 2023-06-27T12:39:03.406 INFO:tasks.rados.rados.0.smithi138.stdout:310: seq_num 173 ranges {507611=777843,2073973=444721} 2023-06-27T12:39:03.412 INFO:tasks.rados.rados.0.smithi138.stdout:310: writing smithi138138488-32 from 507611 to 1285454 tid 1 2023-06-27T12:39:03.419 INFO:tasks.rados.rados.0.smithi138.stdout:310: writing smithi138138488-32 from 2073973 to 2518694 tid 2 2023-06-27T12:39:03.419 INFO:tasks.rados.rados.0.smithi138.stdout:311: read oid 36 snap 28 2023-06-27T12:39:03.420 INFO:tasks.rados.rados.0.smithi138.stdout:311: expect deleted 2023-06-27T12:39:03.420 INFO:tasks.rados.rados.0.smithi138.stdout:312: snap_create 2023-06-27T12:39:03.495 INFO:tasks.rados.rados.0.smithi138.stdout:310: finishing write tid 1 to smithi138138488-32 2023-06-27T12:39:03.536 INFO:tasks.rados.rados.0.smithi138.stdout:310: finishing write tid 2 to smithi138138488-32 2023-06-27T12:39:03.536 INFO:tasks.rados.rados.0.smithi138.stdout:310: finishing write tid 3 to smithi138138488-32 2023-06-27T12:39:03.537 INFO:tasks.rados.rados.0.smithi138.stdout:310: finishing write tid 4 to smithi138138488-32 2023-06-27T12:39:03.537 INFO:tasks.rados.rados.0.smithi138.stdout:310: oid 32 updating version 0 to 97 2023-06-27T12:39:03.537 INFO:tasks.rados.rados.0.smithi138.stdout:310: oid 32 version 97 is already newer than 97 2023-06-27T12:39:03.537 INFO:tasks.rados.rados.0.smithi138.stdout:310: oid 32 version 97 is already newer than 97 2023-06-27T12:39:03.537 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 97 (ObjNum 173 snap 29 seq_num 173) dirty exists 2023-06-27T12:39:03.537 INFO:tasks.rados.rados.0.smithi138.stdout:310: left oid 32 (ObjNum 173 snap 29 seq_num 173) 2023-06-27T12:39:04.430 INFO:tasks.rados.rados.0.smithi138.stdout:310: done (2 left) 2023-06-27T12:39:04.430 INFO:tasks.rados.rados.0.smithi138.stdout:311: done (1 left) 2023-06-27T12:39:04.430 INFO:tasks.rados.rados.0.smithi138.stdout:312: done (0 left) 2023-06-27T12:39:04.430 INFO:tasks.rados.rados.0.smithi138.stdout:313: read oid 23 snap -1 2023-06-27T12:39:04.430 INFO:tasks.rados.rados.0.smithi138.stdout:313: expect deleted 2023-06-27T12:39:04.430 INFO:tasks.rados.rados.0.smithi138.stdout:314: snap_create 2023-06-27T12:39:04.921 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:05.060 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:05.055+0000 7fc25e299700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:05.062 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:05.059+0000 7fc25e299700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:05.428 INFO:tasks.rados.rados.0.smithi138.stdout:313: done (1 left) 2023-06-27T12:39:05.429 INFO:tasks.rados.rados.0.smithi138.stdout:314: done (0 left) 2023-06-27T12:39:05.429 INFO:tasks.rados.rados.0.smithi138.stdout:315: snap_remove snap 30 2023-06-27T12:39:06.431 INFO:tasks.rados.rados.0.smithi138.stdout:315: done (0 left) 2023-06-27T12:39:06.432 INFO:tasks.rados.rados.0.smithi138.stdout:316: snap_create 2023-06-27T12:39:07.087 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:07.088 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:07.088 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:07.134 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:07.235 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:07.227+0000 7fc031f70700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:07.239 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:07.235+0000 7fc031f70700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:07.275 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:07.270+0000 7f640ea73700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:07.278 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:07.274+0000 7f640ea73700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:07.431 INFO:tasks.rados.rados.0.smithi138.stdout:316: done (0 left) 2023-06-27T12:39:07.431 INFO:tasks.rados.rados.0.smithi138.stdout:317: read oid 34 snap -1 2023-06-27T12:39:07.431 INFO:tasks.rados.rados.0.smithi138.stdout:317: expect (ObjNum 127 snap 15 seq_num 127) 2023-06-27T12:39:07.432 INFO:tasks.rados.rados.0.smithi138.stdout:318: delete oid 2 current snap is 32 2023-06-27T12:39:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:07.453 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:07.470 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-06-27T12:39:07.426600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":101364711,"num_objects":111,"num_object_clones":61,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":12,"num_read":621,"num_read_kb":327850,"num_write":730,"num_write_kb":171377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":760,"ondisk_log_size":760,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":417044,"kb_used_data":309748,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377070316,"statfs":{"total":386547056640,"available":386120003584,"internally_reserved":0,"allocated":317181952,"data_stored":333588272,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3327786,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":6,"num_read_kb":4257,"num_write":14,"num_write_kb":3754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.026648"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.812472+0000","last_change":"2023-06-27T12:39:03.816402+0000","last_active":"2023-06-27T12:39:03.812472+0000","last_peered":"2023-06-27T12:39:03.812472+0000","last_clean":"2023-06-27T12:39:03.812472+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:03.812472+0000","last_undegraded":"2023-06-27T12:39:03.812472+0000","last_fullsized":"2023-06-27T12:39:03.812472+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"72'53","reported_seq":207,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.443652+0000","last_change":"2023-06-27T12:39:03.458258+0000","last_active":"2023-06-27T12:39:03.443652+0000","last_peered":"2023-06-27T12:39:03.443652+0000","last_clean":"2023-06-27T12:39:03.443652+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:03.443652+0000","last_undegraded":"2023-06-27T12:39:03.443652+0000","last_fullsized":"2023-06-27T12:39:03.443652+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11411887,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":24620,"num_write":49,"num_write_kb":13946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"73'53","reported_seq":191,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.426682+0000","last_change":"2023-06-27T12:39:02.463067+0000","last_active":"2023-06-27T12:39:03.426682+0000","last_peered":"2023-06-27T12:39:03.426682+0000","last_clean":"2023-06-27T12:39:03.426682+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:03.426682+0000","last_undegraded":"2023-06-27T12:39:03.426682+0000","last_fullsized":"2023-06-27T12:39:03.426682+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924385,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":33,"num_read_kb":12455,"num_write":47,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.807523+0000","last_change":"2023-06-27T12:39:03.813347+0000","last_active":"2023-06-27T12:39:03.807523+0000","last_peered":"2023-06-27T12:39:03.807523+0000","last_clean":"2023-06-27T12:39:03.807523+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:03.807523+0000","last_undegraded":"2023-06-27T12:39:03.807523+0000","last_fullsized":"2023-06-27T12:39:03.807523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":8347,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"48'17","reported_seq":154,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.432673+0000","last_change":"2023-06-27T12:39:03.463147+0000","last_active":"2023-06-27T12:39:03.432673+0000","last_peered":"2023-06-27T12:39:03.432673+0000","last_clean":"2023-06-27T12:39:03.432673+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:03.432673+0000","last_undegraded":"2023-06-27T12:39:03.432673+0000","last_fullsized":"2023-06-27T12:39:03.432673+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268978,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":14490,"num_write":21,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"72'69","reported_seq":218,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.806287+0000","last_change":"2023-06-27T12:39:03.811668+0000","last_active":"2023-06-27T12:39:03.806287+0000","last_peered":"2023-06-27T12:39:03.806287+0000","last_clean":"2023-06-27T12:39:03.806287+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:03.806287+0000","last_undegraded":"2023-06-27T12:39:03.806287+0000","last_fullsized":"2023-06-27T12:39:03.806287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11983325,"num_objects":12,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":86,"num_read_kb":54116,"num_write":58,"num_write_kb":16310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"69'59","reported_seq":195,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.810655+0000","last_change":"2023-06-27T12:39:03.813724+0000","last_active":"2023-06-27T12:39:03.810655+0000","last_peered":"2023-06-27T12:39:03.810655+0000","last_clean":"2023-06-27T12:39:03.810655+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:03.810655+0000","last_undegraded":"2023-06-27T12:39:03.810655+0000","last_fullsized":"2023-06-27T12:39:03.810655+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8084167,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":17113,"num_write":51,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"65'14","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.427581+0000","last_change":"2023-06-27T12:39:02.412745+0000","last_active":"2023-06-27T12:39:03.427581+0000","last_peered":"2023-06-27T12:39:03.427581+0000","last_clean":"2023-06-27T12:39:03.427581+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:03.427581+0000","last_undegraded":"2023-06-27T12:39:03.427581+0000","last_fullsized":"2023-06-27T12:39:03.427581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2705927,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":25158,"num_write":13,"num_write_kb":3961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.448481+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:03.448481+0000","last_peered":"2023-06-27T12:39:03.448481+0000","last_clean":"2023-06-27T12:39:03.448481+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:03.448481+0000","last_undegraded":"2023-06-27T12:39:03.448481+0000","last_fullsized":"2023-06-27T12:39:03.448481+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":132,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.431196+0000","last_change":"2023-06-27T12:39:02.416469+0000","last_active":"2023-06-27T12:39:03.431196+0000","last_peered":"2023-06-27T12:39:03.431196+0000","last_clean":"2023-06-27T12:39:03.431196+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:03.431196+0000","last_undegraded":"2023-06-27T12:39:03.431196+0000","last_fullsized":"2023-06-27T12:39:03.431196+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":105,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.807506+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:03.807506+0000","last_peered":"2023-06-27T12:39:03.807506+0000","last_clean":"2023-06-27T12:39:03.807506+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:03.807506+0000","last_undegraded":"2023-06-27T12:39:03.807506+0000","last_fullsized":"2023-06-27T12:39:03.807506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"69'41","reported_seq":189,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.450666+0000","last_change":"2023-06-27T12:39:03.459186+0000","last_active":"2023-06-27T12:39:03.450666+0000","last_peered":"2023-06-27T12:39:03.450666+0000","last_clean":"2023-06-27T12:39:03.450666+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:03.450666+0000","last_undegraded":"2023-06-27T12:39:03.450666+0000","last_fullsized":"2023-06-27T12:39:03.450666+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722734,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":20336,"num_write":33,"num_write_kb":9145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.429663+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:03.429663+0000","last_peered":"2023-06-27T12:39:03.429663+0000","last_clean":"2023-06-27T12:39:03.429663+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:03.429663+0000","last_undegraded":"2023-06-27T12:39:03.429663+0000","last_fullsized":"2023-06-27T12:39:03.429663+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"75'97","reported_seq":237,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.486537+0000","last_change":"2023-06-27T12:39:03.470422+0000","last_active":"2023-06-27T12:39:03.486537+0000","last_peered":"2023-06-27T12:39:03.486537+0000","last_clean":"2023-06-27T12:39:03.486537+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:03.486537+0000","last_undegraded":"2023-06-27T12:39:03.486537+0000","last_fullsized":"2023-06-27T12:39:03.486537+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14739673,"num_objects":12,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":47873,"num_write":87,"num_write_kb":25580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":109,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.429350+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:03.429350+0000","last_peered":"2023-06-27T12:39:03.429350+0000","last_clean":"2023-06-27T12:39:03.429350+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:03.429350+0000","last_undegraded":"2023-06-27T12:39:03.429350+0000","last_fullsized":"2023-06-27T12:39:03.429350+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":232,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.447538+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:03.447538+0000","last_peered":"2023-06-27T12:39:03.447538+0000","last_clean":"2023-06-27T12:39:03.447538+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:03.447538+0000","last_undegraded":"2023-06-27T12:39:03.447538+0000","last_fullsized":"2023-06-27T12:39:03.447538+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"68'73","reported_seq":227,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.430761+0000","last_change":"2023-06-27T12:39:02.410842+0000","last_active":"2023-06-27T12:39:03.430761+0000","last_peered":"2023-06-27T12:39:03.430761+0000","last_clean":"2023-06-27T12:39:03.430761+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:03.430761+0000","last_undegraded":"2023-06-27T12:39:03.430761+0000","last_fullsized":"2023-06-27T12:39:03.430761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13344692,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":71,"num_read_kb":27019,"num_write":69,"num_write_kb":19399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":105,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.809527+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:03.809527+0000","last_peered":"2023-06-27T12:39:03.809527+0000","last_clean":"2023-06-27T12:39:03.809527+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:03.809527+0000","last_undegraded":"2023-06-27T12:39:03.809527+0000","last_fullsized":"2023-06-27T12:39:03.809527+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'32","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.435835+0000","last_change":"2023-06-27T12:39:03.455960+0000","last_active":"2023-06-27T12:39:03.435835+0000","last_peered":"2023-06-27T12:39:03.435835+0000","last_clean":"2023-06-27T12:39:03.435835+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:03.435835+0000","last_undegraded":"2023-06-27T12:39:03.435835+0000","last_fullsized":"2023-06-27T12:39:03.435835+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3966419,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.453700+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:03.453700+0000","last_peered":"2023-06-27T12:39:03.453700+0000","last_clean":"2023-06-27T12:39:03.453700+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:03.453700+0000","last_undegraded":"2023-06-27T12:39:03.453700+0000","last_fullsized":"2023-06-27T12:39:03.453700+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"68'46","reported_seq":195,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.445851+0000","last_change":"2023-06-27T12:39:03.470858+0000","last_active":"2023-06-27T12:39:03.445851+0000","last_peered":"2023-06-27T12:39:03.445851+0000","last_clean":"2023-06-27T12:39:03.445851+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:03.445851+0000","last_undegraded":"2023-06-27T12:39:03.445851+0000","last_fullsized":"2023-06-27T12:39:03.445851+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7428628,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":27,"num_read_kb":21411,"num_write":41,"num_write_kb":12814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"69'28","reported_seq":157,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.446870+0000","last_change":"2023-06-27T12:39:03.460746+0000","last_active":"2023-06-27T12:39:03.446870+0000","last_peered":"2023-06-27T12:39:03.446870+0000","last_clean":"2023-06-27T12:39:03.446870+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:03.446870+0000","last_undegraded":"2023-06-27T12:39:03.446870+0000","last_fullsized":"2023-06-27T12:39:03.446870+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5109328,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":13727,"num_write":25,"num_write_kb":6934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.451833+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:03.451833+0000","last_peered":"2023-06-27T12:39:03.451833+0000","last_clean":"2023-06-27T12:39:03.451833+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:03.451833+0000","last_undegraded":"2023-06-27T12:39:03.451833+0000","last_fullsized":"2023-06-27T12:39:03.451833+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":109,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.429683+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:03.429683+0000","last_peered":"2023-06-27T12:39:03.429683+0000","last_clean":"2023-06-27T12:39:03.429683+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:03.429683+0000","last_undegraded":"2023-06-27T12:39:03.429683+0000","last_fullsized":"2023-06-27T12:39:03.429683+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"55'41","reported_seq":190,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-27T12:39:03.448783+0000","last_change":"2023-06-27T12:39:03.458106+0000","last_active":"2023-06-27T12:39:03.448783+0000","last_peered":"2023-06-27T12:39:03.448783+0000","last_clean":"2023-06-27T12:39:03.448783+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:03.448783+0000","last_undegraded":"2023-06-27T12:39:03.448783+0000","last_fullsized":"2023-06-27T12:39:03.448783+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7949678,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":22468,"num_write":35,"num_write_kb":10153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":100774343,"num_objects":109,"num_object_clones":61,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":12,"num_read":518,"num_read_kb":327759,"num_write":593,"num_write_kb":169483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":663,"ondisk_log_size":663,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107484,"kb_used_data":80660,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94264356,"statfs":{"total":96636764160,"available":96526700544,"internally_reserved":0,"allocated":82595840,"data_stored":86163968,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103124,"kb_used_data":76300,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94268716,"statfs":{"total":96636764160,"available":96531165184,"internally_reserved":0,"allocated":78131200,"data_stored":81386937,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705735,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98660,"kb_used_data":71836,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94273180,"statfs":{"total":96636764160,"available":96535736320,"internally_reserved":0,"allocated":73560064,"data_stored":78466762,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107776,"kb_used_data":80952,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94264064,"statfs":{"total":96636764160,"available":96526401536,"internally_reserved":0,"allocated":82894848,"data_stored":87570605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:07.475 INFO:tasks.rados.rados.0.smithi138.stdout:317: expect (ObjNum 127 snap 15 seq_num 127) 2023-06-27T12:39:07.573 INFO:tasks.rados.rados.0.smithi138.stdout:317: done (1 left) 2023-06-27T12:39:07.573 INFO:tasks.rados.rados.0.smithi138.stdout:318: done (0 left) 2023-06-27T12:39:07.573 INFO:tasks.rados.rados.0.smithi138.stdout:319: snap_create 2023-06-27T12:39:08.444 INFO:tasks.rados.rados.0.smithi138.stdout:319: done (0 left) 2023-06-27T12:39:08.444 INFO:tasks.rados.rados.0.smithi138.stdout:320: read oid 2 snap -1 2023-06-27T12:39:08.444 INFO:tasks.rados.rados.0.smithi138.stdout:320: expect deleted 2023-06-27T12:39:08.445 INFO:tasks.rados.rados.0.smithi138.stdout:321: write (excl) oid 16 current snap is 33 2023-06-27T12:39:08.445 INFO:tasks.rados.rados.0.smithi138.stdout:321: seq_num 175 ranges {541225=644694,1907290=448390} 2023-06-27T12:39:08.451 INFO:tasks.rados.rados.0.smithi138.stdout:321: writing smithi138138488-16 from 541225 to 1185919 tid 1 2023-06-27T12:39:08.458 INFO:tasks.rados.rados.0.smithi138.stdout:321: writing smithi138138488-16 from 1907290 to 2355680 tid 2 2023-06-27T12:39:08.458 INFO:tasks.rados.rados.0.smithi138.stdout:322: write oid 13 current snap is 33 2023-06-27T12:39:08.458 INFO:tasks.rados.rados.0.smithi138.stdout:322: seq_num 176 ranges {558032=719943,1849419=592048} 2023-06-27T12:39:08.468 INFO:tasks.rados.rados.0.smithi138.stdout:322: writing smithi138138488-13 from 558032 to 1277975 tid 1 2023-06-27T12:39:08.476 INFO:tasks.rados.rados.0.smithi138.stdout:322: writing smithi138138488-13 from 1849419 to 2441467 tid 2 2023-06-27T12:39:08.477 INFO:tasks.rados.rados.0.smithi138.stdout:323: delete oid 22 current snap is 33 2023-06-27T12:39:08.526 INFO:tasks.rados.rados.0.smithi138.stdout:322: finishing write tid 1 to smithi138138488-13 2023-06-27T12:39:08.564 INFO:tasks.rados.rados.0.smithi138.stdout:323: done (3 left) 2023-06-27T12:39:08.565 INFO:tasks.rados.rados.0.smithi138.stdout:324: delete oid 35 current snap is 33 2023-06-27T12:39:08.568 INFO:tasks.rados.rados.0.smithi138.stdout:322: finishing write tid 2 to smithi138138488-13 2023-06-27T12:39:08.573 INFO:tasks.rados.rados.0.smithi138.stdout:321: finishing write tid 1 to smithi138138488-16 2023-06-27T12:39:08.609 INFO:tasks.rados.rados.0.smithi138.stdout:322: finishing write tid 3 to smithi138138488-13 2023-06-27T12:39:08.609 INFO:tasks.rados.rados.0.smithi138.stdout:322: finishing write tid 4 to smithi138138488-13 2023-06-27T12:39:08.609 INFO:tasks.rados.rados.0.smithi138.stdout:322: oid 13 updating version 0 to 103 2023-06-27T12:39:08.609 INFO:tasks.rados.rados.0.smithi138.stdout:322: oid 13 version 103 is already newer than 103 2023-06-27T12:39:08.610 INFO:tasks.rados.rados.0.smithi138.stdout:322: oid 13 version 103 is already newer than 103 2023-06-27T12:39:08.610 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 103 (ObjNum 176 snap 33 seq_num 176) dirty exists 2023-06-27T12:39:08.610 INFO:tasks.rados.rados.0.smithi138.stdout:322: left oid 13 (ObjNum 176 snap 33 seq_num 176) 2023-06-27T12:39:08.610 INFO:tasks.rados.rados.0.smithi138.stdout:320: done (3 left) 2023-06-27T12:39:08.610 INFO:tasks.rados.rados.0.smithi138.stdout:322: done (2 left) 2023-06-27T12:39:08.610 INFO:tasks.rados.rados.0.smithi138.stdout:324: done (1 left) 2023-06-27T12:39:08.611 INFO:tasks.rados.rados.0.smithi138.stdout:325: delete oid 10 current snap is 33 2023-06-27T12:39:08.621 INFO:tasks.rados.rados.0.smithi138.stdout:321: finishing write tid 2 to smithi138138488-16 2023-06-27T12:39:08.662 INFO:tasks.rados.rados.0.smithi138.stdout:321: finishing write tid 3 to smithi138138488-16 2023-06-27T12:39:08.662 INFO:tasks.rados.rados.0.smithi138.stdout:321: finishing write tid 4 to smithi138138488-16 2023-06-27T12:39:08.662 INFO:tasks.rados.rados.0.smithi138.stdout:321: oid 16 updating version 0 to 72 2023-06-27T12:39:08.662 INFO:tasks.rados.rados.0.smithi138.stdout:321: oid 16 version 72 is already newer than 70 2023-06-27T12:39:08.662 INFO:tasks.rados.rados.0.smithi138.stdout:321: oid 16 version 72 is already newer than 72 2023-06-27T12:39:08.662 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 72 (ObjNum 175 snap 33 seq_num 175) dirty exists 2023-06-27T12:39:08.663 INFO:tasks.rados.rados.0.smithi138.stdout:321: left oid 16 (ObjNum 175 snap 33 seq_num 175) 2023-06-27T12:39:08.663 INFO:tasks.rados.rados.0.smithi138.stdout:321: done (1 left) 2023-06-27T12:39:08.663 INFO:tasks.rados.rados.0.smithi138.stdout:325: done (0 left) 2023-06-27T12:39:08.663 INFO:tasks.rados.rados.0.smithi138.stdout:326: write oid 26 current snap is 33 2023-06-27T12:39:08.663 INFO:tasks.rados.rados.0.smithi138.stdout:326: seq_num 180 ranges {625260=620939,1683347=599386,2943668=81592} 2023-06-27T12:39:08.668 INFO:tasks.rados.rados.0.smithi138.stdout:326: writing smithi138138488-26 from 625260 to 1246199 tid 1 2023-06-27T12:39:08.674 INFO:tasks.rados.rados.0.smithi138.stdout:326: writing smithi138138488-26 from 1683347 to 2282733 tid 2 2023-06-27T12:39:08.678 INFO:tasks.rados.rados.0.smithi138.stdout:326: writing smithi138138488-26 from 2943668 to 3025260 tid 3 2023-06-27T12:39:08.678 INFO:tasks.rados.rados.0.smithi138.stdout:327: snap_remove snap 29 2023-06-27T12:39:09.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:09.449 INFO:tasks.rados.rados.0.smithi138.stdout:327: done (1 left) 2023-06-27T12:39:09.449 INFO:tasks.rados.rados.0.smithi138.stdout:328: snap_create 2023-06-27T12:39:09.449 INFO:tasks.rados.rados.0.smithi138.stdout:326: finishing write tid 1 to smithi138138488-26 2023-06-27T12:39:09.449 INFO:tasks.rados.rados.0.smithi138.stdout:326: finishing write tid 2 to smithi138138488-26 2023-06-27T12:39:09.449 INFO:tasks.rados.rados.0.smithi138.stdout:326: finishing write tid 3 to smithi138138488-26 2023-06-27T12:39:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:326: finishing write tid 4 to smithi138138488-26 2023-06-27T12:39:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:326: finishing write tid 5 to smithi138138488-26 2023-06-27T12:39:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:326: oid 26 updating version 0 to 50 2023-06-27T12:39:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:326: oid 26 version 50 is already newer than 50 2023-06-27T12:39:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:326: oid 26 version 50 is already newer than 50 2023-06-27T12:39:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:326: oid 26 version 50 is already newer than 50 2023-06-27T12:39:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 50 (ObjNum 180 snap 33 seq_num 180) dirty exists 2023-06-27T12:39:09.451 INFO:tasks.rados.rados.0.smithi138.stdout:326: left oid 26 (ObjNum 180 snap 33 seq_num 180) 2023-06-27T12:39:09.479 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:09.475+0000 7f961a42f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:09.482 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:09.479+0000 7f961a42f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:10.381 INFO:tasks.rados.rados.0.smithi138.stdout:326: done (1 left) 2023-06-27T12:39:10.381 INFO:tasks.rados.rados.0.smithi138.stdout:328: done (0 left) 2023-06-27T12:39:10.381 INFO:tasks.rados.rados.0.smithi138.stdout:329: write (excl) oid 29 current snap is 34 2023-06-27T12:39:10.381 INFO:tasks.rados.rados.0.smithi138.stdout:329: seq_num 181 ranges {642067=696188,2025476=459397,2971091=70976} 2023-06-27T12:39:10.390 INFO:tasks.rados.rados.0.smithi138.stdout:329: writing smithi138138488-29 from 642067 to 1338255 tid 1 2023-06-27T12:39:10.399 INFO:tasks.rados.rados.0.smithi138.stdout:329: writing smithi138138488-29 from 2025476 to 2484873 tid 2 2023-06-27T12:39:10.403 INFO:tasks.rados.rados.0.smithi138.stdout:329: writing smithi138138488-29 from 2971091 to 3042067 tid 3 2023-06-27T12:39:10.403 INFO:tasks.rados.rados.0.smithi138.stdout:330: snap_remove snap 25 2023-06-27T12:39:11.398 INFO:tasks.rados.rados.0.smithi138.stdout:330: done (1 left) 2023-06-27T12:39:11.398 INFO:tasks.rados.rados.0.smithi138.stdout:331: write (excl) oid 28 current snap is 34 2023-06-27T12:39:11.398 INFO:tasks.rados.rados.0.smithi138.stdout:329: finishing write tid 1 to smithi138138488-29 2023-06-27T12:39:11.399 INFO:tasks.rados.rados.0.smithi138.stdout:331: seq_num 182 ranges {658874=771437,2083958=603055} 2023-06-27T12:39:11.405 INFO:tasks.rados.rados.0.smithi138.stdout:331: writing smithi138138488-28 from 658874 to 1430311 tid 1 2023-06-27T12:39:11.413 INFO:tasks.rados.rados.0.smithi138.stdout:331: writing smithi138138488-28 from 2083958 to 2687013 tid 2 2023-06-27T12:39:11.414 INFO:tasks.rados.rados.0.smithi138.stdout:332: write oid 19 current snap is 34 2023-06-27T12:39:11.414 INFO:tasks.rados.rados.0.smithi138.stdout:329: finishing write tid 2 to smithi138138488-29 2023-06-27T12:39:11.414 INFO:tasks.rados.rados.0.smithi138.stdout:332: seq_num 183 ranges {675681=563039,1858793=746713,3025937=49744} 2023-06-27T12:39:11.421 INFO:tasks.rados.rados.0.smithi138.stdout:332: writing smithi138138488-19 from 675681 to 1238720 tid 1 2023-06-27T12:39:11.432 INFO:tasks.rados.rados.0.smithi138.stdout:332: writing smithi138138488-19 from 1858793 to 2605506 tid 2 2023-06-27T12:39:11.435 INFO:tasks.rados.rados.0.smithi138.stdout:332: writing smithi138138488-19 from 3025937 to 3075681 tid 3 2023-06-27T12:39:11.436 INFO:tasks.rados.rados.0.smithi138.stdout:333: read oid 41 snap 32 2023-06-27T12:39:11.436 INFO:tasks.rados.rados.0.smithi138.stdout:333: expect (ObjNum 113 snap 10 seq_num 113) 2023-06-27T12:39:11.436 INFO:tasks.rados.rados.0.smithi138.stdout:334: snap_remove snap 33 2023-06-27T12:39:11.552 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:11.673+0000 7f3d66314700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:11.680 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:11.677+0000 7f3d66314700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:12.399 INFO:tasks.rados.rados.0.smithi138.stdout:334: done (4 left) 2023-06-27T12:39:12.399 INFO:tasks.rados.rados.0.smithi138.stdout:335: write oid 18 current snap is 34 2023-06-27T12:39:12.399 INFO:tasks.rados.rados.0.smithi138.stdout:329: finishing write tid 3 to smithi138138488-29 2023-06-27T12:39:12.400 INFO:tasks.rados.rados.0.smithi138.stdout:335: seq_num 184 ranges {692488=638288,1917275=606724} 2023-06-27T12:39:12.406 INFO:tasks.rados.rados.0.smithi138.stdout:335: writing smithi138138488-18 from 692488 to 1330776 tid 1 2023-06-27T12:39:12.414 INFO:tasks.rados.rados.0.smithi138.stdout:335: writing smithi138138488-18 from 1917275 to 2523999 tid 2 2023-06-27T12:39:12.414 INFO:tasks.rados.rados.0.smithi138.stdout:336: read oid 47 snap -1 2023-06-27T12:39:12.415 INFO:tasks.rados.rados.0.smithi138.stdout:336: expect (ObjNum 46 snap 0 seq_num 46) 2023-06-27T12:39:12.415 INFO:tasks.rados.rados.0.smithi138.stdout:337: write (excl) oid 17 current snap is 34 2023-06-27T12:39:12.415 INFO:tasks.rados.rados.0.smithi138.stdout:329: finishing write tid 4 to smithi138138488-29 2023-06-27T12:39:12.416 INFO:tasks.rados.rados.0.smithi138.stdout:337: seq_num 185 ranges {709295=713537,1859404=750382} 2023-06-27T12:39:12.424 INFO:tasks.rados.rados.0.smithi138.stdout:337: writing smithi138138488-17 from 709295 to 1422832 tid 1 2023-06-27T12:39:12.434 INFO:tasks.rados.rados.0.smithi138.stdout:337: writing smithi138138488-17 from 1859404 to 2609786 tid 2 2023-06-27T12:39:12.434 INFO:tasks.rados.rados.0.smithi138.stdout:338: snap_remove snap 28 2023-06-27T12:39:12.473 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:12.473 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:12.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:12.607 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:12.600+0000 7f3e32700700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:12.610 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:12.607+0000 7f3e32700700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:12.802 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:12.812 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:12.816 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2023-06-27T12:39:12.394446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":95398325,"num_objects":117,"num_object_clones":66,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":15,"num_read":634,"num_read_kb":332446,"num_write":743,"num_write_kb":175001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":773,"ondisk_log_size":773,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":428092,"kb_used_data":320796,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377059268,"statfs":{"total":386547056640,"available":386108690432,"internally_reserved":0,"allocated":328495104,"data_stored":347247690,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5966386,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":13,"num_read_kb":4596,"num_write":13,"num_write_kb":3624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.955003"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":140,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.486762+0000","last_change":"2023-06-27T12:39:07.457706+0000","last_active":"2023-06-27T12:39:08.486762+0000","last_peered":"2023-06-27T12:39:08.486762+0000","last_clean":"2023-06-27T12:39:08.486762+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:08.486762+0000","last_undegraded":"2023-06-27T12:39:08.486762+0000","last_fullsized":"2023-06-27T12:39:08.486762+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"72'53","reported_seq":214,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.520155+0000","last_change":"2023-06-27T12:39:07.900006+0000","last_active":"2023-06-27T12:39:08.520155+0000","last_peered":"2023-06-27T12:39:08.520155+0000","last_clean":"2023-06-27T12:39:08.520155+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:08.520155+0000","last_undegraded":"2023-06-27T12:39:08.520155+0000","last_fullsized":"2023-06-27T12:39:08.520155+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11411887,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":24620,"num_write":49,"num_write_kb":13946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"73'53","reported_seq":200,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.474716+0000","last_change":"2023-06-27T12:39:06.445409+0000","last_active":"2023-06-27T12:39:08.474716+0000","last_peered":"2023-06-27T12:39:08.474716+0000","last_clean":"2023-06-27T12:39:08.474716+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:08.474716+0000","last_undegraded":"2023-06-27T12:39:08.474716+0000","last_fullsized":"2023-06-27T12:39:08.474716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924385,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":33,"num_read_kb":12455,"num_write":47,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":178,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.468790+0000","last_change":"2023-06-27T12:39:07.461237+0000","last_active":"2023-06-27T12:39:08.468790+0000","last_peered":"2023-06-27T12:39:08.468790+0000","last_clean":"2023-06-27T12:39:08.468790+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:08.468790+0000","last_undegraded":"2023-06-27T12:39:08.468790+0000","last_fullsized":"2023-06-27T12:39:08.468790+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":8347,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"48'17","reported_seq":161,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.457223+0000","last_change":"2023-06-27T12:39:07.463047+0000","last_active":"2023-06-27T12:39:08.457223+0000","last_peered":"2023-06-27T12:39:08.457223+0000","last_clean":"2023-06-27T12:39:08.457223+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:08.457223+0000","last_undegraded":"2023-06-27T12:39:08.457223+0000","last_fullsized":"2023-06-27T12:39:08.457223+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268978,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":14490,"num_write":21,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"80'72","reported_seq":232,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.572619+0000","last_change":"2023-06-27T12:39:07.463395+0000","last_active":"2023-06-27T12:39:08.572619+0000","last_peered":"2023-06-27T12:39:08.572619+0000","last_clean":"2023-06-27T12:39:08.572619+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:08.572619+0000","last_undegraded":"2023-06-27T12:39:08.572619+0000","last_fullsized":"2023-06-27T12:39:08.572619+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14924550,"num_objects":13,"num_object_clones":7,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":95,"num_read_kb":58710,"num_write":62,"num_write_kb":17378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"69'59","reported_seq":203,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.472821+0000","last_change":"2023-06-27T12:39:07.461472+0000","last_active":"2023-06-27T12:39:08.472821+0000","last_peered":"2023-06-27T12:39:08.472821+0000","last_clean":"2023-06-27T12:39:08.472821+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:08.472821+0000","last_undegraded":"2023-06-27T12:39:08.472821+0000","last_fullsized":"2023-06-27T12:39:08.472821+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5630345,"num_objects":11,"num_object_clones":8,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":38,"num_read_kb":17113,"num_write":51,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"65'14","reported_seq":168,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.479109+0000","last_change":"2023-06-27T12:39:06.447227+0000","last_active":"2023-06-27T12:39:08.479109+0000","last_peered":"2023-06-27T12:39:08.479109+0000","last_clean":"2023-06-27T12:39:08.479109+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:08.479109+0000","last_undegraded":"2023-06-27T12:39:08.479109+0000","last_fullsized":"2023-06-27T12:39:08.479109+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2705927,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":25158,"num_write":13,"num_write_kb":3961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.505916+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:08.505916+0000","last_peered":"2023-06-27T12:39:08.505916+0000","last_clean":"2023-06-27T12:39:08.505916+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:08.505916+0000","last_undegraded":"2023-06-27T12:39:08.505916+0000","last_fullsized":"2023-06-27T12:39:08.505916+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.475832+0000","last_change":"2023-06-27T12:39:06.445974+0000","last_active":"2023-06-27T12:39:08.475832+0000","last_peered":"2023-06-27T12:39:08.475832+0000","last_clean":"2023-06-27T12:39:08.475832+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:08.475832+0000","last_undegraded":"2023-06-27T12:39:08.475832+0000","last_fullsized":"2023-06-27T12:39:08.475832+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":111,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.483965+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:08.483965+0000","last_peered":"2023-06-27T12:39:08.483965+0000","last_clean":"2023-06-27T12:39:08.483965+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:08.483965+0000","last_undegraded":"2023-06-27T12:39:08.483965+0000","last_fullsized":"2023-06-27T12:39:08.483965+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"69'41","reported_seq":196,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.523364+0000","last_change":"2023-06-27T12:39:07.899667+0000","last_active":"2023-06-27T12:39:08.523364+0000","last_peered":"2023-06-27T12:39:08.523364+0000","last_clean":"2023-06-27T12:39:08.523364+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:08.523364+0000","last_undegraded":"2023-06-27T12:39:08.523364+0000","last_fullsized":"2023-06-27T12:39:08.523364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722734,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":20336,"num_write":33,"num_write_kb":9145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":117,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.479453+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:08.479453+0000","last_peered":"2023-06-27T12:39:08.479453+0000","last_clean":"2023-06-27T12:39:08.479453+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:08.479453+0000","last_undegraded":"2023-06-27T12:39:08.479453+0000","last_fullsized":"2023-06-27T12:39:08.479453+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"80'103","reported_seq":250,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.567122+0000","last_change":"2023-06-27T12:39:07.463589+0000","last_active":"2023-06-27T12:39:08.567122+0000","last_peered":"2023-06-27T12:39:08.567122+0000","last_clean":"2023-06-27T12:39:08.567122+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:08.567122+0000","last_undegraded":"2023-06-27T12:39:08.567122+0000","last_fullsized":"2023-06-27T12:39:08.567122+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403500,"num_objects":15,"num_object_clones":10,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":60,"num_read_kb":47874,"num_write":91,"num_write_kb":26863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":117,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.478446+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:08.478446+0000","last_peered":"2023-06-27T12:39:08.478446+0000","last_clean":"2023-06-27T12:39:08.478446+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:08.478446+0000","last_undegraded":"2023-06-27T12:39:08.478446+0000","last_fullsized":"2023-06-27T12:39:08.478446+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":238,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.448911+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:08.448911+0000","last_peered":"2023-06-27T12:39:08.448911+0000","last_clean":"2023-06-27T12:39:08.448911+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:08.448911+0000","last_undegraded":"2023-06-27T12:39:08.448911+0000","last_fullsized":"2023-06-27T12:39:08.448911+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"68'73","reported_seq":236,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.480731+0000","last_change":"2023-06-27T12:39:06.449245+0000","last_active":"2023-06-27T12:39:08.480731+0000","last_peered":"2023-06-27T12:39:08.480731+0000","last_clean":"2023-06-27T12:39:08.480731+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:08.480731+0000","last_undegraded":"2023-06-27T12:39:08.480731+0000","last_fullsized":"2023-06-27T12:39:08.480731+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13344692,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":71,"num_read_kb":27019,"num_write":69,"num_write_kb":19399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":111,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.471307+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:08.471307+0000","last_peered":"2023-06-27T12:39:08.471307+0000","last_clean":"2023-06-27T12:39:08.471307+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:08.471307+0000","last_undegraded":"2023-06-27T12:39:08.471307+0000","last_fullsized":"2023-06-27T12:39:08.471307+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'32","reported_seq":174,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.454731+0000","last_change":"2023-06-27T12:39:07.465675+0000","last_active":"2023-06-27T12:39:08.454731+0000","last_peered":"2023-06-27T12:39:08.454731+0000","last_clean":"2023-06-27T12:39:08.454731+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:08.454731+0000","last_undegraded":"2023-06-27T12:39:08.454731+0000","last_fullsized":"2023-06-27T12:39:08.454731+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3966419,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.453812+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:08.453812+0000","last_peered":"2023-06-27T12:39:08.453812+0000","last_clean":"2023-06-27T12:39:08.453812+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:08.453812+0000","last_undegraded":"2023-06-27T12:39:08.453812+0000","last_fullsized":"2023-06-27T12:39:08.453812+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"80'50","reported_seq":207,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.692358+0000","last_change":"2023-06-27T12:39:07.902288+0000","last_active":"2023-06-27T12:39:08.692358+0000","last_peered":"2023-06-27T12:39:08.692358+0000","last_clean":"2023-06-27T12:39:08.692358+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:08.692358+0000","last_undegraded":"2023-06-27T12:39:08.692358+0000","last_fullsized":"2023-06-27T12:39:08.692358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10453888,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":28,"num_read_kb":21412,"num_write":46,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"69'28","reported_seq":164,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.452874+0000","last_change":"2023-06-27T12:39:07.465261+0000","last_active":"2023-06-27T12:39:08.452874+0000","last_peered":"2023-06-27T12:39:08.452874+0000","last_clean":"2023-06-27T12:39:08.452874+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:08.452874+0000","last_undegraded":"2023-06-27T12:39:08.452874+0000","last_fullsized":"2023-06-27T12:39:08.452874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5109328,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":13727,"num_write":25,"num_write_kb":6934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.509034+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:08.509034+0000","last_peered":"2023-06-27T12:39:08.509034+0000","last_clean":"2023-06-27T12:39:08.509034+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:08.509034+0000","last_undegraded":"2023-06-27T12:39:08.509034+0000","last_fullsized":"2023-06-27T12:39:08.509034+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":117,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.467199+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:08.467199+0000","last_peered":"2023-06-27T12:39:08.467199+0000","last_clean":"2023-06-27T12:39:08.467199+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:08.467199+0000","last_undegraded":"2023-06-27T12:39:08.467199+0000","last_fullsized":"2023-06-27T12:39:08.467199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"55'41","reported_seq":198,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-06-27T12:39:08.504559+0000","last_change":"2023-06-27T12:39:07.889564+0000","last_active":"2023-06-27T12:39:08.504559+0000","last_peered":"2023-06-27T12:39:08.504559+0000","last_clean":"2023-06-27T12:39:08.504559+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:08.504559+0000","last_undegraded":"2023-06-27T12:39:08.504559+0000","last_fullsized":"2023-06-27T12:39:08.504559+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4806802,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":27,"num_read_kb":22468,"num_write":35,"num_write_kb":10153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":94807957,"num_objects":115,"num_object_clones":66,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":15,"num_read":531,"num_read_kb":332355,"num_write":606,"num_write_kb":173107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":676,"ondisk_log_size":676,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110084,"kb_used_data":83260,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94261756,"statfs":{"total":96636764160,"available":96524038144,"internally_reserved":0,"allocated":85258240,"data_stored":89618563,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":106800,"kb_used_data":79976,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94265040,"statfs":{"total":96636764160,"available":96527400960,"internally_reserved":0,"allocated":81895424,"data_stored":85934616,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705740,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101044,"kb_used_data":74220,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94270796,"statfs":{"total":96636764160,"available":96533295104,"internally_reserved":0,"allocated":76001280,"data_stored":80878144,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110164,"kb_used_data":83340,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94261676,"statfs":{"total":96636764160,"available":96523956224,"internally_reserved":0,"allocated":85340160,"data_stored":90816367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:13.401 INFO:tasks.rados.rados.0.smithi138.stdout:338: done (7 left) 2023-06-27T12:39:13.401 INFO:tasks.rados.rados.0.smithi138.stdout:339: read oid 45 snap -1 2023-06-27T12:39:13.401 INFO:tasks.rados.rados.0.smithi138.stdout:339: expect (ObjNum 151 snap 21 seq_num 151) 2023-06-27T12:39:13.401 INFO:tasks.rados.rados.0.smithi138.stdout:340: snap_create 2023-06-27T12:39:13.402 INFO:tasks.rados.rados.0.smithi138.stdout:329: finishing write tid 5 to smithi138138488-29 2023-06-27T12:39:13.402 INFO:tasks.rados.rados.0.smithi138.stdout:329: oid 29 updating version 0 to 35 2023-06-27T12:39:13.402 INFO:tasks.rados.rados.0.smithi138.stdout:329: oid 29 version 35 is already newer than 31 2023-06-27T12:39:13.402 INFO:tasks.rados.rados.0.smithi138.stdout:329: oid 29 version 35 is already newer than 35 2023-06-27T12:39:13.402 INFO:tasks.rados.rados.0.smithi138.stdout:329: oid 29 version 35 is already newer than 35 2023-06-27T12:39:13.402 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 35 (ObjNum 181 snap 34 seq_num 181) dirty exists 2023-06-27T12:39:13.402 INFO:tasks.rados.rados.0.smithi138.stdout:329: left oid 29 (ObjNum 181 snap 34 seq_num 181) 2023-06-27T12:39:13.403 INFO:tasks.rados.rados.0.smithi138.stdout:332: finishing write tid 1 to smithi138138488-19 2023-06-27T12:39:13.403 INFO:tasks.rados.rados.0.smithi138.stdout:331: finishing write tid 1 to smithi138138488-28 2023-06-27T12:39:13.403 INFO:tasks.rados.rados.0.smithi138.stdout:332: finishing write tid 2 to smithi138138488-19 2023-06-27T12:39:13.403 INFO:tasks.rados.rados.0.smithi138.stdout:332: finishing write tid 3 to smithi138138488-19 2023-06-27T12:39:13.403 INFO:tasks.rados.rados.0.smithi138.stdout:332: finishing write tid 4 to smithi138138488-19 2023-06-27T12:39:13.403 INFO:tasks.rados.rados.0.smithi138.stdout:332: finishing write tid 5 to smithi138138488-19 2023-06-27T12:39:13.403 INFO:tasks.rados.rados.0.smithi138.stdout:332: oid 19 updating version 0 to 78 2023-06-27T12:39:13.404 INFO:tasks.rados.rados.0.smithi138.stdout:332: oid 19 version 78 is already newer than 78 2023-06-27T12:39:13.404 INFO:tasks.rados.rados.0.smithi138.stdout:332: oid 19 version 78 is already newer than 78 2023-06-27T12:39:13.404 INFO:tasks.rados.rados.0.smithi138.stdout:332: oid 19 version 78 is already newer than 78 2023-06-27T12:39:13.404 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 78 (ObjNum 183 snap 34 seq_num 183) dirty exists 2023-06-27T12:39:13.405 INFO:tasks.rados.rados.0.smithi138.stdout:332: left oid 19 (ObjNum 183 snap 34 seq_num 183) 2023-06-27T12:39:13.405 INFO:tasks.rados.rados.0.smithi138.stdout:333: expect (ObjNum 113 snap 10 seq_num 113) 2023-06-27T12:39:13.604 INFO:tasks.rados.rados.0.smithi138.stdout:331: finishing write tid 2 to smithi138138488-28 2023-06-27T12:39:13.605 INFO:tasks.rados.rados.0.smithi138.stdout:331: finishing write tid 3 to smithi138138488-28 2023-06-27T12:39:13.605 INFO:tasks.rados.rados.0.smithi138.stdout:331: finishing write tid 4 to smithi138138488-28 2023-06-27T12:39:13.605 INFO:tasks.rados.rados.0.smithi138.stdout:331: oid 28 updating version 0 to 75 2023-06-27T12:39:13.605 INFO:tasks.rados.rados.0.smithi138.stdout:331: oid 28 version 75 is already newer than 75 2023-06-27T12:39:13.605 INFO:tasks.rados.rados.0.smithi138.stdout:331: oid 28 version 75 is already newer than 68 2023-06-27T12:39:13.605 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 75 (ObjNum 182 snap 34 seq_num 182) dirty exists 2023-06-27T12:39:13.605 INFO:tasks.rados.rados.0.smithi138.stdout:331: left oid 28 (ObjNum 182 snap 34 seq_num 182) 2023-06-27T12:39:13.606 INFO:tasks.rados.rados.0.smithi138.stdout:336: expect (ObjNum 46 snap 0 seq_num 46) 2023-06-27T12:39:13.708 INFO:tasks.rados.rados.0.smithi138.stdout:335: finishing write tid 1 to smithi138138488-18 2023-06-27T12:39:13.708 INFO:tasks.rados.rados.0.smithi138.stdout:335: finishing write tid 2 to smithi138138488-18 2023-06-27T12:39:13.708 INFO:tasks.rados.rados.0.smithi138.stdout:335: finishing write tid 3 to smithi138138488-18 2023-06-27T12:39:13.708 INFO:tasks.rados.rados.0.smithi138.stdout:335: finishing write tid 4 to smithi138138488-18 2023-06-27T12:39:13.709 INFO:tasks.rados.rados.0.smithi138.stdout:335: oid 18 updating version 0 to 59 2023-06-27T12:39:13.709 INFO:tasks.rados.rados.0.smithi138.stdout:335: oid 18 version 59 is already newer than 59 2023-06-27T12:39:13.709 INFO:tasks.rados.rados.0.smithi138.stdout:335: oid 18 version 59 is already newer than 59 2023-06-27T12:39:13.709 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 59 (ObjNum 184 snap 34 seq_num 184) dirty exists 2023-06-27T12:39:13.709 INFO:tasks.rados.rados.0.smithi138.stdout:335: left oid 18 (ObjNum 184 snap 34 seq_num 184) 2023-06-27T12:39:13.709 INFO:tasks.rados.rados.0.smithi138.stdout:337: finishing write tid 1 to smithi138138488-17 2023-06-27T12:39:13.709 INFO:tasks.rados.rados.0.smithi138.stdout:337: finishing write tid 2 to smithi138138488-17 2023-06-27T12:39:13.710 INFO:tasks.rados.rados.0.smithi138.stdout:337: finishing write tid 3 to smithi138138488-17 2023-06-27T12:39:13.710 INFO:tasks.rados.rados.0.smithi138.stdout:337: finishing write tid 4 to smithi138138488-17 2023-06-27T12:39:13.710 INFO:tasks.rados.rados.0.smithi138.stdout:337: oid 17 updating version 0 to 57 2023-06-27T12:39:13.710 INFO:tasks.rados.rados.0.smithi138.stdout:337: oid 17 version 57 is already newer than 54 2023-06-27T12:39:13.710 INFO:tasks.rados.rados.0.smithi138.stdout:337: oid 17 version 57 is already newer than 57 2023-06-27T12:39:13.711 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 57 (ObjNum 185 snap 34 seq_num 185) dirty exists 2023-06-27T12:39:13.711 INFO:tasks.rados.rados.0.smithi138.stdout:337: left oid 17 (ObjNum 185 snap 34 seq_num 185) 2023-06-27T12:39:13.711 INFO:tasks.rados.rados.0.smithi138.stdout:339: expect (ObjNum 151 snap 21 seq_num 151) 2023-06-27T12:39:13.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:13.878 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:13.873+0000 7f6e25df2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:13.881 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:13.877+0000 7f6e25df2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:14.402 INFO:tasks.rados.rados.0.smithi138.stdout:329: done (8 left) 2023-06-27T12:39:14.402 INFO:tasks.rados.rados.0.smithi138.stdout:331: done (7 left) 2023-06-27T12:39:14.402 INFO:tasks.rados.rados.0.smithi138.stdout:332: done (6 left) 2023-06-27T12:39:14.402 INFO:tasks.rados.rados.0.smithi138.stdout:333: done (5 left) 2023-06-27T12:39:14.402 INFO:tasks.rados.rados.0.smithi138.stdout:335: done (4 left) 2023-06-27T12:39:14.402 INFO:tasks.rados.rados.0.smithi138.stdout:336: done (3 left) 2023-06-27T12:39:14.403 INFO:tasks.rados.rados.0.smithi138.stdout:337: done (2 left) 2023-06-27T12:39:14.403 INFO:tasks.rados.rados.0.smithi138.stdout:339: done (1 left) 2023-06-27T12:39:14.403 INFO:tasks.rados.rados.0.smithi138.stdout:340: done (0 left) 2023-06-27T12:39:14.403 INFO:tasks.rados.rados.0.smithi138.stdout:341: snap_create 2023-06-27T12:39:15.406 INFO:tasks.rados.rados.0.smithi138.stdout:341: done (0 left) 2023-06-27T12:39:15.407 INFO:tasks.rados.rados.0.smithi138.stdout:342: read oid 24 snap 32 2023-06-27T12:39:15.407 INFO:tasks.rados.rados.0.smithi138.stdout:342: expect (ObjNum 73 snap 3 seq_num 73) 2023-06-27T12:39:15.408 INFO:tasks.rados.rados.0.smithi138.stdout:343: delete oid 27 current snap is 36 2023-06-27T12:39:15.446 INFO:tasks.rados.rados.0.smithi138.stdout:342: expect (ObjNum 73 snap 3 seq_num 73) 2023-06-27T12:39:15.606 INFO:tasks.rados.rados.0.smithi138.stdout:342: done (1 left) 2023-06-27T12:39:15.606 INFO:tasks.rados.rados.0.smithi138.stdout:343: done (0 left) 2023-06-27T12:39:15.607 INFO:tasks.rados.rados.0.smithi138.stdout:344: write (excl) oid 14 current snap is 36 2023-06-27T12:39:15.607 INFO:tasks.rados.rados.0.smithi138.stdout:344: seq_num 187 ranges {742909=464035,1976368=754051} 2023-06-27T12:39:15.612 INFO:tasks.rados.rados.0.smithi138.stdout:344: writing smithi138138488-14 from 742909 to 1206944 tid 1 2023-06-27T12:39:15.623 INFO:tasks.rados.rados.0.smithi138.stdout:344: writing smithi138138488-14 from 1976368 to 2730419 tid 2 2023-06-27T12:39:15.623 INFO:tasks.rados.rados.0.smithi138.stdout:345: snap_create 2023-06-27T12:39:15.655 INFO:tasks.rados.rados.0.smithi138.stdout:344: finishing write tid 1 to smithi138138488-14 2023-06-27T12:39:15.709 INFO:tasks.rados.rados.0.smithi138.stdout:344: finishing write tid 2 to smithi138138488-14 2023-06-27T12:39:15.710 INFO:tasks.rados.rados.0.smithi138.stdout:344: finishing write tid 3 to smithi138138488-14 2023-06-27T12:39:15.710 INFO:tasks.rados.rados.0.smithi138.stdout:344: finishing write tid 4 to smithi138138488-14 2023-06-27T12:39:15.711 INFO:tasks.rados.rados.0.smithi138.stdout:344: oid 14 updating version 0 to 84 2023-06-27T12:39:15.711 INFO:tasks.rados.rados.0.smithi138.stdout:344: oid 14 version 84 is already newer than 81 2023-06-27T12:39:15.711 INFO:tasks.rados.rados.0.smithi138.stdout:344: oid 14 version 84 is already newer than 84 2023-06-27T12:39:15.711 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 84 (ObjNum 187 snap 36 seq_num 187) dirty exists 2023-06-27T12:39:15.711 INFO:tasks.rados.rados.0.smithi138.stdout:344: left oid 14 (ObjNum 187 snap 36 seq_num 187) 2023-06-27T12:39:15.953 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:16.078 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:16.073+0000 7ff836ad5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:16.081 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:16.078+0000 7ff836ad5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:16.467 INFO:tasks.rados.rados.0.smithi138.stdout:344: done (1 left) 2023-06-27T12:39:16.467 INFO:tasks.rados.rados.0.smithi138.stdout:345: done (0 left) 2023-06-27T12:39:16.468 INFO:tasks.rados.rados.0.smithi138.stdout:346: write (excl) oid 13 current snap is 37 2023-06-27T12:39:16.468 INFO:tasks.rados.rados.0.smithi138.stdout:346: seq_num 188 ranges {759716=539284,2034850=614062} 2023-06-27T12:39:16.474 INFO:tasks.rados.rados.0.smithi138.stdout:346: writing smithi138138488-13 from 759716 to 1299000 tid 1 2023-06-27T12:39:16.481 INFO:tasks.rados.rados.0.smithi138.stdout:346: writing smithi138138488-13 from 2034850 to 2648912 tid 2 2023-06-27T12:39:16.482 INFO:tasks.rados.rados.0.smithi138.stdout:347: snap_create 2023-06-27T12:39:16.554 INFO:tasks.rados.rados.0.smithi138.stdout:346: finishing write tid 1 to smithi138138488-13 2023-06-27T12:39:16.601 INFO:tasks.rados.rados.0.smithi138.stdout:346: finishing write tid 2 to smithi138138488-13 2023-06-27T12:39:16.604 INFO:tasks.rados.rados.0.smithi138.stdout:346: finishing write tid 3 to smithi138138488-13 2023-06-27T12:39:16.606 INFO:tasks.rados.rados.0.smithi138.stdout:346: finishing write tid 4 to smithi138138488-13 2023-06-27T12:39:16.607 INFO:tasks.rados.rados.0.smithi138.stdout:346: oid 13 updating version 0 to 125 2023-06-27T12:39:16.607 INFO:tasks.rados.rados.0.smithi138.stdout:346: oid 13 version 125 is already newer than 125 2023-06-27T12:39:16.607 INFO:tasks.rados.rados.0.smithi138.stdout:346: oid 13 version 125 is already newer than 122 2023-06-27T12:39:16.607 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 125 (ObjNum 188 snap 37 seq_num 188) dirty exists 2023-06-27T12:39:16.608 INFO:tasks.rados.rados.0.smithi138.stdout:346: left oid 13 (ObjNum 188 snap 37 seq_num 188) 2023-06-27T12:39:17.470 INFO:tasks.rados.rados.0.smithi138.stdout:346: done (1 left) 2023-06-27T12:39:17.471 INFO:tasks.rados.rados.0.smithi138.stdout:347: done (0 left) 2023-06-27T12:39:17.471 INFO:tasks.rados.rados.0.smithi138.stdout:348: read oid 49 snap -1 2023-06-27T12:39:17.471 INFO:tasks.rados.rados.0.smithi138.stdout:348: expect (ObjNum 142 snap 18 seq_num 142) 2023-06-27T12:39:17.471 INFO:tasks.rados.rados.0.smithi138.stdout:349: read oid 33 snap 37 2023-06-27T12:39:17.472 INFO:tasks.rados.rados.0.smithi138.stdout:349: expect (ObjNum 124 snap 15 seq_num 124) 2023-06-27T12:39:17.472 INFO:tasks.rados.rados.0.smithi138.stdout:350: read oid 38 snap -1 2023-06-27T12:39:17.472 INFO:tasks.rados.rados.0.smithi138.stdout:350: expect deleted 2023-06-27T12:39:17.472 INFO:tasks.rados.rados.0.smithi138.stdout:351: snap_remove snap 35 2023-06-27T12:39:17.819 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:17.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:17.819 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:17.965 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:17.959+0000 7f7a7dc0f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:17.968 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:17.964+0000 7f7a7dc0f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:18.153 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:18.167 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:18.169 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:18.181 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2023-06-27T12:39:17.921253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97011830,"num_objects":122,"num_object_clones":71,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":15,"num_read":648,"num_read_kb":347405,"num_write":765,"num_write_kb":181519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":815,"ondisk_log_size":815,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":439852,"kb_used_data":332556,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377047508,"statfs":{"total":386547056640,"available":386096648192,"internally_reserved":0,"allocated":340537344,"data_stored":363621932,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1613505,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14959,"num_write":22,"num_write_kb":6518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000854"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-06-27T12:39:11.428381+0000","last_change":"2023-06-27T12:39:11.431054+0000","last_active":"2023-06-27T12:39:11.428381+0000","last_peered":"2023-06-27T12:39:11.428381+0000","last_clean":"2023-06-27T12:39:11.428381+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:11.428381+0000","last_undegraded":"2023-06-27T12:39:11.428381+0000","last_fullsized":"2023-06-27T12:39:11.428381+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"84'57","reported_seq":235,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.441007+0000","last_change":"2023-06-27T12:39:13.441710+0000","last_active":"2023-06-27T12:39:13.441007+0000","last_peered":"2023-06-27T12:39:13.441007+0000","last_clean":"2023-06-27T12:39:13.441007+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:13.441007+0000","last_undegraded":"2023-06-27T12:39:13.441007+0000","last_fullsized":"2023-06-27T12:39:13.441007+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11647185,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":28513,"num_write":53,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'59","reported_seq":220,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.416875+0000","last_change":"2023-06-27T12:39:13.417029+0000","last_active":"2023-06-27T12:39:13.416875+0000","last_peered":"2023-06-27T12:39:13.416875+0000","last_clean":"2023-06-27T12:39:13.416875+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:13.416875+0000","last_undegraded":"2023-06-27T12:39:13.416875+0000","last_fullsized":"2023-06-27T12:39:13.416875+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3092488,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":34,"num_read_kb":12456,"num_write":51,"num_write_kb":13729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":186,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-06-27T12:39:11.415637+0000","last_change":"2023-06-27T12:39:11.448347+0000","last_active":"2023-06-27T12:39:11.415637+0000","last_peered":"2023-06-27T12:39:11.415637+0000","last_clean":"2023-06-27T12:39:11.415637+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:11.415637+0000","last_undegraded":"2023-06-27T12:39:11.415637+0000","last_fullsized":"2023-06-27T12:39:11.415637+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":8347,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"48'17","reported_seq":173,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.446081+0000","last_change":"2023-06-27T12:39:13.461081+0000","last_active":"2023-06-27T12:39:13.446081+0000","last_peered":"2023-06-27T12:39:13.446081+0000","last_clean":"2023-06-27T12:39:13.446081+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:13.446081+0000","last_undegraded":"2023-06-27T12:39:13.446081+0000","last_fullsized":"2023-06-27T12:39:13.446081+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268978,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":14490,"num_write":21,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"80'72","reported_seq":241,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-06-27T12:39:11.425095+0000","last_change":"2023-06-27T12:39:11.425775+0000","last_active":"2023-06-27T12:39:11.425095+0000","last_peered":"2023-06-27T12:39:11.425095+0000","last_clean":"2023-06-27T12:39:11.425095+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:11.425095+0000","last_undegraded":"2023-06-27T12:39:11.425095+0000","last_fullsized":"2023-06-27T12:39:11.425095+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14924550,"num_objects":13,"num_object_clones":7,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":95,"num_read_kb":58710,"num_write":62,"num_write_kb":17378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":215,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-06-27T12:39:11.556422+0000","last_change":"2023-06-27T12:39:11.558081+0000","last_active":"2023-06-27T12:39:11.556422+0000","last_peered":"2023-06-27T12:39:11.556422+0000","last_clean":"2023-06-27T12:39:11.556422+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:11.556422+0000","last_undegraded":"2023-06-27T12:39:11.556422+0000","last_fullsized":"2023-06-27T12:39:11.556422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":40,"num_read_kb":17114,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"65'14","reported_seq":182,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.416490+0000","last_change":"2023-06-27T12:39:13.421569+0000","last_active":"2023-06-27T12:39:13.416490+0000","last_peered":"2023-06-27T12:39:13.416490+0000","last_clean":"2023-06-27T12:39:13.416490+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:13.416490+0000","last_undegraded":"2023-06-27T12:39:13.416490+0000","last_fullsized":"2023-06-27T12:39:13.416490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2705927,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":25158,"num_write":13,"num_write_kb":3961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.422000+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:13.422000+0000","last_peered":"2023-06-27T12:39:13.422000+0000","last_clean":"2023-06-27T12:39:13.422000+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:13.422000+0000","last_undegraded":"2023-06-27T12:39:13.422000+0000","last_fullsized":"2023-06-27T12:39:13.422000+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":155,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.412022+0000","last_change":"2023-06-27T12:39:13.415496+0000","last_active":"2023-06-27T12:39:13.412022+0000","last_peered":"2023-06-27T12:39:13.412022+0000","last_clean":"2023-06-27T12:39:13.412022+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:13.412022+0000","last_undegraded":"2023-06-27T12:39:13.412022+0000","last_fullsized":"2023-06-27T12:39:13.412022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":117,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-06-27T12:39:11.424270+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:11.424270+0000","last_peered":"2023-06-27T12:39:11.424270+0000","last_clean":"2023-06-27T12:39:11.424270+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:11.424270+0000","last_undegraded":"2023-06-27T12:39:11.424270+0000","last_fullsized":"2023-06-27T12:39:11.424270+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"69'41","reported_seq":210,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.418457+0000","last_change":"2023-06-27T12:39:13.439847+0000","last_active":"2023-06-27T12:39:13.418457+0000","last_peered":"2023-06-27T12:39:13.418457+0000","last_clean":"2023-06-27T12:39:13.418457+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:13.418457+0000","last_undegraded":"2023-06-27T12:39:13.418457+0000","last_fullsized":"2023-06-27T12:39:13.418457+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722734,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":20336,"num_write":33,"num_write_kb":9145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":127,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.411193+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:13.411193+0000","last_peered":"2023-06-27T12:39:13.411193+0000","last_clean":"2023-06-27T12:39:13.411193+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:13.411193+0000","last_undegraded":"2023-06-27T12:39:13.411193+0000","last_fullsized":"2023-06-27T12:39:13.411193+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"80'105","reported_seq":261,"reported_epoch":85,"state":"active+clean+snaptrim","last_fresh":"2023-06-27T12:39:13.463519+0000","last_change":"2023-06-27T12:39:12.424893+0000","last_active":"2023-06-27T12:39:13.463519+0000","last_peered":"2023-06-27T12:39:13.463519+0000","last_clean":"2023-06-27T12:39:13.463519+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:13.463519+0000","last_undegraded":"2023-06-27T12:39:13.463519+0000","last_fullsized":"2023-06-27T12:39:13.463519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403500,"num_objects":15,"num_object_clones":10,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":62,"num_read_kb":52832,"num_write":91,"num_write_kb":26863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":127,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.409663+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:13.409663+0000","last_peered":"2023-06-27T12:39:13.409663+0000","last_clean":"2023-06-27T12:39:13.409663+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:13.409663+0000","last_undegraded":"2023-06-27T12:39:13.409663+0000","last_fullsized":"2023-06-27T12:39:13.409663+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":246,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.432810+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:13.432810+0000","last_peered":"2023-06-27T12:39:13.432810+0000","last_clean":"2023-06-27T12:39:13.432810+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:13.432810+0000","last_undegraded":"2023-06-27T12:39:13.432810+0000","last_fullsized":"2023-06-27T12:39:13.432810+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"83'78","reported_seq":256,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.414514+0000","last_change":"2023-06-27T12:39:13.414747+0000","last_active":"2023-06-27T12:39:13.414514+0000","last_peered":"2023-06-27T12:39:13.414514+0000","last_clean":"2023-06-27T12:39:13.414514+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:13.414514+0000","last_undegraded":"2023-06-27T12:39:13.414514+0000","last_fullsized":"2023-06-27T12:39:13.414514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14117814,"num_objects":13,"num_object_clones":7,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":72,"num_read_kb":27020,"num_write":74,"num_write_kb":20728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":117,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-06-27T12:39:11.419993+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:11.419993+0000","last_peered":"2023-06-27T12:39:11.419993+0000","last_clean":"2023-06-27T12:39:11.419993+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:11.419993+0000","last_undegraded":"2023-06-27T12:39:11.419993+0000","last_fullsized":"2023-06-27T12:39:11.419993+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'32","reported_seq":186,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.417362+0000","last_change":"2023-06-27T12:39:13.435356+0000","last_active":"2023-06-27T12:39:13.417362+0000","last_peered":"2023-06-27T12:39:13.417362+0000","last_clean":"2023-06-27T12:39:13.417362+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:13.417362+0000","last_undegraded":"2023-06-27T12:39:13.417362+0000","last_fullsized":"2023-06-27T12:39:13.417362+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3966419,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.432415+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:13.432415+0000","last_peered":"2023-06-27T12:39:13.432415+0000","last_clean":"2023-06-27T12:39:13.432415+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:13.432415+0000","last_undegraded":"2023-06-27T12:39:13.432415+0000","last_fullsized":"2023-06-27T12:39:13.432415+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"80'50","reported_seq":223,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.420294+0000","last_change":"2023-06-27T12:39:13.424611+0000","last_active":"2023-06-27T12:39:13.420294+0000","last_peered":"2023-06-27T12:39:13.420294+0000","last_clean":"2023-06-27T12:39:13.420294+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:13.420294+0000","last_undegraded":"2023-06-27T12:39:13.420294+0000","last_fullsized":"2023-06-27T12:39:13.420294+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10453888,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":30,"num_read_kb":27516,"num_write":46,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"82'35","reported_seq":178,"reported_epoch":85,"state":"active+clean+snaptrim","last_fresh":"2023-06-27T12:39:13.427440+0000","last_change":"2023-06-27T12:39:12.431556+0000","last_active":"2023-06-27T12:39:13.427440+0000","last_peered":"2023-06-27T12:39:13.427440+0000","last_clean":"2023-06-27T12:39:13.427440+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:13.427440+0000","last_undegraded":"2023-06-27T12:39:13.427440+0000","last_fullsized":"2023-06-27T12:39:13.427440+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5327819,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":13728,"num_write":30,"num_write_kb":8133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.419783+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:13.419783+0000","last_peered":"2023-06-27T12:39:13.419783+0000","last_clean":"2023-06-27T12:39:13.419783+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:13.419783+0000","last_undegraded":"2023-06-27T12:39:13.419783+0000","last_fullsized":"2023-06-27T12:39:13.419783+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.417829+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:13.417829+0000","last_peered":"2023-06-27T12:39:13.417829+0000","last_clean":"2023-06-27T12:39:13.417829+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:13.417829+0000","last_undegraded":"2023-06-27T12:39:13.417829+0000","last_fullsized":"2023-06-27T12:39:13.417829+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"80'43","reported_seq":212,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-06-27T12:39:13.416538+0000","last_change":"2023-06-27T12:39:13.505477+0000","last_active":"2023-06-27T12:39:13.416538+0000","last_peered":"2023-06-27T12:39:13.416538+0000","last_clean":"2023-06-27T12:39:13.416538+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:13.416538+0000","last_undegraded":"2023-06-27T12:39:13.416538+0000","last_fullsized":"2023-06-27T12:39:13.416538+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4806802,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":27,"num_read_kb":22468,"num_write":35,"num_write_kb":10153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":96421462,"num_objects":120,"num_object_clones":71,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":15,"num_read":545,"num_read_kb":347314,"num_write":628,"num_write_kb":179625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":718,"ondisk_log_size":718,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673040,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113836,"kb_used_data":87012,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94258004,"statfs":{"total":96636764160,"available":96520196096,"internally_reserved":0,"allocated":89100288,"data_stored":94818038,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673040,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":107548,"kb_used_data":80724,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94264292,"statfs":{"total":96636764160,"available":96526635008,"internally_reserved":0,"allocated":82661376,"data_stored":87468009,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":103512,"kb_used_data":76688,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94268328,"statfs":{"total":96636764160,"available":96530767872,"internally_reserved":0,"allocated":78528512,"data_stored":84176817,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673040,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":114956,"kb_used_data":88132,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94256884,"statfs":{"total":96636764160,"available":96519049216,"internally_reserved":0,"allocated":90247168,"data_stored":97159068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:18.279 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:18.275+0000 7f41285cc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:18.282 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:18.279+0000 7f41285cc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:18.471 INFO:tasks.rados.rados.0.smithi138.stdout:351: done (3 left) 2023-06-27T12:39:18.471 INFO:tasks.rados.rados.0.smithi138.stdout:352: snap_remove snap 31 2023-06-27T12:39:19.472 INFO:tasks.rados.rados.0.smithi138.stdout:352: done (3 left) 2023-06-27T12:39:19.473 INFO:tasks.rados.rados.0.smithi138.stdout:353: snap_remove snap 36 2023-06-27T12:39:20.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:20.386 INFO:tasks.rados.rados.0.smithi138.stdout:353: done (3 left) 2023-06-27T12:39:20.386 INFO:tasks.rados.rados.0.smithi138.stdout:354: delete oid 15 current snap is 38 2023-06-27T12:39:20.386 INFO:tasks.rados.rados.0.smithi138.stdout:348: expect (ObjNum 142 snap 18 seq_num 142) 2023-06-27T12:39:20.483 INFO:tasks.rados.rados.0.smithi138.stdout:349: expect (ObjNum 124 snap 15 seq_num 124) 2023-06-27T12:39:20.498 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:20.492+0000 7efcdec9c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:20.502 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:20.498+0000 7efcdec9c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:20.598 INFO:tasks.rados.rados.0.smithi138.stdout:348: done (3 left) 2023-06-27T12:39:20.599 INFO:tasks.rados.rados.0.smithi138.stdout:349: done (2 left) 2023-06-27T12:39:20.599 INFO:tasks.rados.rados.0.smithi138.stdout:350: done (1 left) 2023-06-27T12:39:20.599 INFO:tasks.rados.rados.0.smithi138.stdout:354: done (0 left) 2023-06-27T12:39:20.599 INFO:tasks.rados.rados.0.smithi138.stdout:355: write (excl) oid 24 current snap is 38 2023-06-27T12:39:20.599 INFO:tasks.rados.rados.0.smithi138.stdout:355: seq_num 190 ranges {793330=689782,2035461=617731,3101545=91785} 2023-06-27T12:39:20.606 INFO:tasks.rados.rados.0.smithi138.stdout:355: writing smithi138138488-24 from 793330 to 1483112 tid 1 2023-06-27T12:39:20.615 INFO:tasks.rados.rados.0.smithi138.stdout:355: writing smithi138138488-24 from 2035461 to 2653192 tid 2 2023-06-27T12:39:20.619 INFO:tasks.rados.rados.0.smithi138.stdout:355: writing smithi138138488-24 from 3101545 to 3193330 tid 3 2023-06-27T12:39:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:356: read oid 7 snap -1 2023-06-27T12:39:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:356: expect (ObjNum 166 snap 26 seq_num 166) 2023-06-27T12:39:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:357: read oid 14 snap -1 2023-06-27T12:39:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:357: expect (ObjNum 187 snap 36 seq_num 187) 2023-06-27T12:39:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:358: write oid 36 current snap is 38 2023-06-27T12:39:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:358: seq_num 191 ranges {410137=481384,1410296=761389,2728968=481169} 2023-06-27T12:39:20.626 INFO:tasks.rados.rados.0.smithi138.stdout:358: writing smithi138138488-36 from 410137 to 891521 tid 1 2023-06-27T12:39:20.637 INFO:tasks.rados.rados.0.smithi138.stdout:358: writing smithi138138488-36 from 1410296 to 2171685 tid 2 2023-06-27T12:39:20.645 INFO:tasks.rados.rados.0.smithi138.stdout:358: writing smithi138138488-36 from 2728968 to 3210137 tid 3 2023-06-27T12:39:20.645 INFO:tasks.rados.rados.0.smithi138.stdout:359: read oid 4 snap -1 2023-06-27T12:39:20.645 INFO:tasks.rados.rados.0.smithi138.stdout:359: expect (ObjNum 117 snap 10 seq_num 117) 2023-06-27T12:39:20.645 INFO:tasks.rados.rados.0.smithi138.stdout:360: read oid 45 snap -1 2023-06-27T12:39:20.645 INFO:tasks.rados.rados.0.smithi138.stdout:360: expect (ObjNum 151 snap 21 seq_num 151) 2023-06-27T12:39:20.646 INFO:tasks.rados.rados.0.smithi138.stdout:361: write oid 47 current snap is 38 2023-06-27T12:39:20.646 INFO:tasks.rados.rados.0.smithi138.stdout:361: seq_num 192 ranges {426944=556633,1468778=621400,2872744=354200} 2023-06-27T12:39:20.653 INFO:tasks.rados.rados.0.smithi138.stdout:361: writing smithi138138488-47 from 426944 to 983577 tid 1 2023-06-27T12:39:20.662 INFO:tasks.rados.rados.0.smithi138.stdout:361: writing smithi138138488-47 from 1468778 to 2090178 tid 2 2023-06-27T12:39:20.670 INFO:tasks.rados.rados.0.smithi138.stdout:361: writing smithi138138488-47 from 2872744 to 3226944 tid 3 2023-06-27T12:39:20.670 INFO:tasks.rados.rados.0.smithi138.stdout:362: snap_create 2023-06-27T12:39:20.723 INFO:tasks.rados.rados.0.smithi138.stdout:356: expect (ObjNum 166 snap 26 seq_num 166) 2023-06-27T12:39:20.949 INFO:tasks.rados.rados.0.smithi138.stdout:357: expect (ObjNum 187 snap 36 seq_num 187) 2023-06-27T12:39:21.150 INFO:tasks.rados.rados.0.smithi138.stdout:361: finishing write tid 1 to smithi138138488-47 2023-06-27T12:39:21.151 INFO:tasks.rados.rados.0.smithi138.stdout:355: finishing write tid 1 to smithi138138488-24 2023-06-27T12:39:21.151 INFO:tasks.rados.rados.0.smithi138.stdout:361: finishing write tid 2 to smithi138138488-47 2023-06-27T12:39:21.151 INFO:tasks.rados.rados.0.smithi138.stdout:358: finishing write tid 1 to smithi138138488-36 2023-06-27T12:39:21.151 INFO:tasks.rados.rados.0.smithi138.stdout:358: finishing write tid 2 to smithi138138488-36 2023-06-27T12:39:21.152 INFO:tasks.rados.rados.0.smithi138.stdout:360: expect (ObjNum 151 snap 21 seq_num 151) 2023-06-27T12:39:21.301 INFO:tasks.rados.rados.0.smithi138.stdout:359: expect (ObjNum 117 snap 10 seq_num 117) 2023-06-27T12:39:21.506 INFO:tasks.rados.rados.0.smithi138.stdout:358: finishing write tid 3 to smithi138138488-36 2023-06-27T12:39:21.507 INFO:tasks.rados.rados.0.smithi138.stdout:358: finishing write tid 4 to smithi138138488-36 2023-06-27T12:39:21.507 INFO:tasks.rados.rados.0.smithi138.stdout:358: finishing write tid 5 to smithi138138488-36 2023-06-27T12:39:21.507 INFO:tasks.rados.rados.0.smithi138.stdout:358: oid 36 updating version 0 to 63 2023-06-27T12:39:21.507 INFO:tasks.rados.rados.0.smithi138.stdout:358: oid 36 version 63 is already newer than 63 2023-06-27T12:39:21.507 INFO:tasks.rados.rados.0.smithi138.stdout:358: oid 36 version 63 is already newer than 63 2023-06-27T12:39:21.508 INFO:tasks.rados.rados.0.smithi138.stdout:358: oid 36 version 63 is already newer than 63 2023-06-27T12:39:21.508 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 63 (ObjNum 191 snap 38 seq_num 191) dirty exists 2023-06-27T12:39:21.508 INFO:tasks.rados.rados.0.smithi138.stdout:358: left oid 36 (ObjNum 191 snap 38 seq_num 191) 2023-06-27T12:39:21.508 INFO:tasks.rados.rados.0.smithi138.stdout:361: finishing write tid 3 to smithi138138488-47 2023-06-27T12:39:21.508 INFO:tasks.rados.rados.0.smithi138.stdout:361: finishing write tid 4 to smithi138138488-47 2023-06-27T12:39:21.508 INFO:tasks.rados.rados.0.smithi138.stdout:361: finishing write tid 5 to smithi138138488-47 2023-06-27T12:39:21.508 INFO:tasks.rados.rados.0.smithi138.stdout:361: oid 47 updating version 0 to 66 2023-06-27T12:39:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:361: oid 47 version 66 is already newer than 66 2023-06-27T12:39:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:361: oid 47 version 66 is already newer than 66 2023-06-27T12:39:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:361: oid 47 version 66 is already newer than 66 2023-06-27T12:39:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 47 v 66 (ObjNum 192 snap 38 seq_num 192) dirty exists 2023-06-27T12:39:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:361: left oid 47 (ObjNum 192 snap 38 seq_num 192) 2023-06-27T12:39:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:355: finishing write tid 2 to smithi138138488-24 2023-06-27T12:39:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:356: done (7 left) 2023-06-27T12:39:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:357: done (6 left) 2023-06-27T12:39:21.510 INFO:tasks.rados.rados.0.smithi138.stdout:358: done (5 left) 2023-06-27T12:39:21.510 INFO:tasks.rados.rados.0.smithi138.stdout:359: done (4 left) 2023-06-27T12:39:21.510 INFO:tasks.rados.rados.0.smithi138.stdout:360: done (3 left) 2023-06-27T12:39:21.510 INFO:tasks.rados.rados.0.smithi138.stdout:361: done (2 left) 2023-06-27T12:39:21.510 INFO:tasks.rados.rados.0.smithi138.stdout:362: done (1 left) 2023-06-27T12:39:21.510 INFO:tasks.rados.rados.0.smithi138.stdout:363: snap_remove snap 34 2023-06-27T12:39:22.498 INFO:tasks.rados.rados.0.smithi138.stdout:363: done (1 left) 2023-06-27T12:39:22.499 INFO:tasks.rados.rados.0.smithi138.stdout:364: snap_create 2023-06-27T12:39:22.499 INFO:tasks.rados.rados.0.smithi138.stdout:355: finishing write tid 3 to smithi138138488-24 2023-06-27T12:39:22.499 INFO:tasks.rados.rados.0.smithi138.stdout:355: finishing write tid 4 to smithi138138488-24 2023-06-27T12:39:22.499 INFO:tasks.rados.rados.0.smithi138.stdout:355: finishing write tid 5 to smithi138138488-24 2023-06-27T12:39:22.500 INFO:tasks.rados.rados.0.smithi138.stdout:355: oid 24 updating version 0 to 77 2023-06-27T12:39:22.500 INFO:tasks.rados.rados.0.smithi138.stdout:355: oid 24 version 77 is already newer than 77 2023-06-27T12:39:22.500 INFO:tasks.rados.rados.0.smithi138.stdout:355: oid 24 version 77 is already newer than 77 2023-06-27T12:39:22.501 INFO:tasks.rados.rados.0.smithi138.stdout:355: oid 24 version 77 is already newer than 73 2023-06-27T12:39:22.501 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 77 (ObjNum 190 snap 38 seq_num 190) dirty exists 2023-06-27T12:39:22.501 INFO:tasks.rados.rados.0.smithi138.stdout:355: left oid 24 (ObjNum 190 snap 38 seq_num 190) 2023-06-27T12:39:22.574 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:22.704 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:22.700+0000 7f11af91e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:22.707 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:22.704+0000 7f11af91e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:23.184 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:23.185 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:23.185 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:23.313 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:23.306+0000 7f8196aeb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:23.316 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:23.312+0000 7f8196aeb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:23.499 INFO:tasks.rados.rados.0.smithi138.stdout:355: done (1 left) 2023-06-27T12:39:23.499 INFO:tasks.rados.rados.0.smithi138.stdout:364: done (0 left) 2023-06-27T12:39:23.499 INFO:tasks.rados.rados.0.smithi138.stdout:365: write (excl) oid 4 current snap is 40 2023-06-27T12:39:23.499 INFO:tasks.rados.rados.0.smithi138.stdout:365: seq_num 193 ranges {443751=631882,1810907=765058,3067461=176290} 2023-06-27T12:39:23.508 INFO:tasks.rados.rados.0.smithi138.stdout:365: writing smithi138138488-4 from 443751 to 1075633 tid 1 2023-06-27T12:39:23.521 INFO:tasks.rados.rados.0.smithi138.stdout:365: writing smithi138138488-4 from 1810907 to 2575965 tid 2 2023-06-27T12:39:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:23.526 INFO:tasks.rados.rados.0.smithi138.stdout:365: writing smithi138138488-4 from 3067461 to 3243751 tid 3 2023-06-27T12:39:23.526 INFO:tasks.rados.rados.0.smithi138.stdout:366: write (excl) oid 22 current snap is 40 2023-06-27T12:39:23.526 INFO:tasks.rados.rados.0.smithi138.stdout:366: seq_num 194 ranges {460558=707131,1869389=508716,3094884=165674} 2023-06-27T12:39:23.528 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:23.535 INFO:tasks.rados.rados.0.smithi138.stdout:366: writing smithi138138488-22 from 460558 to 1167689 tid 1 2023-06-27T12:39:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2023-06-27T12:39:23.493019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":94826953,"num_objects":125,"num_object_clones":74,"num_object_copies":375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":16,"num_read":669,"num_read_kb":363613,"num_write":773,"num_write_kb":183837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":843,"ondisk_log_size":843,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":446924,"kb_used_data":339628,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377040436,"statfs":{"total":386547056640,"available":386089406464,"internally_reserved":0,"allocated":347779072,"data_stored":375132551,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025605"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-06-27T12:39:18.480293+0000","last_change":"2023-06-27T12:39:18.484320+0000","last_active":"2023-06-27T12:39:18.480293+0000","last_peered":"2023-06-27T12:39:18.480293+0000","last_clean":"2023-06-27T12:39:18.480293+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:18.480293+0000","last_undegraded":"2023-06-27T12:39:18.480293+0000","last_fullsized":"2023-06-27T12:39:18.480293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"87'59","reported_seq":242,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.483062+0000","last_change":"2023-06-27T12:39:13.441710+0000","last_active":"2023-06-27T12:39:17.483062+0000","last_peered":"2023-06-27T12:39:17.483062+0000","last_clean":"2023-06-27T12:39:17.483062+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:17.483062+0000","last_undegraded":"2023-06-27T12:39:17.483062+0000","last_fullsized":"2023-06-27T12:39:17.483062+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8453888,"num_objects":11,"num_object_clones":7,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":38,"num_read_kb":28513,"num_write":53,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'59","reported_seq":226,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.504408+0000","last_change":"2023-06-27T12:39:13.417029+0000","last_active":"2023-06-27T12:39:17.504408+0000","last_peered":"2023-06-27T12:39:17.504408+0000","last_clean":"2023-06-27T12:39:17.504408+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:17.504408+0000","last_undegraded":"2023-06-27T12:39:17.504408+0000","last_fullsized":"2023-06-27T12:39:17.504408+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3092488,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":34,"num_read_kb":12456,"num_write":51,"num_write_kb":13729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":202,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-06-27T12:39:18.478609+0000","last_change":"2023-06-27T12:39:18.482574+0000","last_active":"2023-06-27T12:39:18.478609+0000","last_peered":"2023-06-27T12:39:18.478609+0000","last_clean":"2023-06-27T12:39:18.478609+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:18.478609+0000","last_undegraded":"2023-06-27T12:39:18.478609+0000","last_fullsized":"2023-06-27T12:39:18.478609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":37,"num_read_kb":13465,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"48'17","reported_seq":179,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.491548+0000","last_change":"2023-06-27T12:39:13.461081+0000","last_active":"2023-06-27T12:39:17.491548+0000","last_peered":"2023-06-27T12:39:17.491548+0000","last_clean":"2023-06-27T12:39:17.491548+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:17.491548+0000","last_undegraded":"2023-06-27T12:39:17.491548+0000","last_fullsized":"2023-06-27T12:39:17.491548+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268978,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":14490,"num_write":21,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"80'72","reported_seq":257,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-06-27T12:39:18.477437+0000","last_change":"2023-06-27T12:39:18.483970+0000","last_active":"2023-06-27T12:39:18.477437+0000","last_peered":"2023-06-27T12:39:18.477437+0000","last_clean":"2023-06-27T12:39:18.477437+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:18.477437+0000","last_undegraded":"2023-06-27T12:39:18.477437+0000","last_fullsized":"2023-06-27T12:39:18.477437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14924550,"num_objects":13,"num_object_clones":7,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":102,"num_read_kb":66646,"num_write":62,"num_write_kb":17378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":228,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-06-27T12:39:18.479666+0000","last_change":"2023-06-27T12:39:18.482849+0000","last_active":"2023-06-27T12:39:18.479666+0000","last_peered":"2023-06-27T12:39:18.479666+0000","last_clean":"2023-06-27T12:39:18.479666+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:18.479666+0000","last_undegraded":"2023-06-27T12:39:18.479666+0000","last_fullsized":"2023-06-27T12:39:18.479666+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":40,"num_read_kb":17114,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"65'14","reported_seq":188,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.500590+0000","last_change":"2023-06-27T12:39:13.421569+0000","last_active":"2023-06-27T12:39:17.500590+0000","last_peered":"2023-06-27T12:39:17.500590+0000","last_clean":"2023-06-27T12:39:17.500590+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:17.500590+0000","last_undegraded":"2023-06-27T12:39:17.500590+0000","last_fullsized":"2023-06-27T12:39:17.500590+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2705927,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":25158,"num_write":13,"num_write_kb":3961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.491080+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:17.491080+0000","last_peered":"2023-06-27T12:39:17.491080+0000","last_clean":"2023-06-27T12:39:17.491080+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:17.491080+0000","last_undegraded":"2023-06-27T12:39:17.491080+0000","last_fullsized":"2023-06-27T12:39:17.491080+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":161,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.502982+0000","last_change":"2023-06-27T12:39:13.415496+0000","last_active":"2023-06-27T12:39:17.502982+0000","last_peered":"2023-06-27T12:39:17.502982+0000","last_clean":"2023-06-27T12:39:17.502982+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:17.502982+0000","last_undegraded":"2023-06-27T12:39:17.502982+0000","last_fullsized":"2023-06-27T12:39:17.502982+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":127,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-06-27T12:39:18.472773+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:18.472773+0000","last_peered":"2023-06-27T12:39:18.472773+0000","last_clean":"2023-06-27T12:39:18.472773+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:18.472773+0000","last_undegraded":"2023-06-27T12:39:18.472773+0000","last_fullsized":"2023-06-27T12:39:18.472773+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"69'41","reported_seq":216,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.493052+0000","last_change":"2023-06-27T12:39:13.439847+0000","last_active":"2023-06-27T12:39:17.493052+0000","last_peered":"2023-06-27T12:39:17.493052+0000","last_clean":"2023-06-27T12:39:17.493052+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:17.493052+0000","last_undegraded":"2023-06-27T12:39:17.493052+0000","last_fullsized":"2023-06-27T12:39:17.493052+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722734,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":20336,"num_write":33,"num_write_kb":9145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.510353+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:17.510353+0000","last_peered":"2023-06-27T12:39:17.510353+0000","last_clean":"2023-06-27T12:39:17.510353+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:17.510353+0000","last_undegraded":"2023-06-27T12:39:17.510353+0000","last_fullsized":"2023-06-27T12:39:17.510353+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"88'125","reported_seq":274,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.496839+0000","last_change":"2023-06-27T12:39:15.440680+0000","last_active":"2023-06-27T12:39:17.496839+0000","last_peered":"2023-06-27T12:39:17.496839+0000","last_clean":"2023-06-27T12:39:17.496839+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:17.496839+0000","last_undegraded":"2023-06-27T12:39:17.496839+0000","last_fullsized":"2023-06-27T12:39:17.496839+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8605184,"num_objects":16,"num_object_clones":11,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":52833,"num_write":95,"num_write_kb":27990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":133,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.498319+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:17.498319+0000","last_peered":"2023-06-27T12:39:17.498319+0000","last_clean":"2023-06-27T12:39:17.498319+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:17.498319+0000","last_undegraded":"2023-06-27T12:39:17.498319+0000","last_fullsized":"2023-06-27T12:39:17.498319+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":252,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.504237+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:17.504237+0000","last_peered":"2023-06-27T12:39:17.504237+0000","last_clean":"2023-06-27T12:39:17.504237+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:17.504237+0000","last_undegraded":"2023-06-27T12:39:17.504237+0000","last_fullsized":"2023-06-27T12:39:17.504237+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"87'84","reported_seq":266,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.500834+0000","last_change":"2023-06-27T12:39:13.414747+0000","last_active":"2023-06-27T12:39:17.500834+0000","last_peered":"2023-06-27T12:39:17.500834+0000","last_clean":"2023-06-27T12:39:17.500834+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:17.500834+0000","last_undegraded":"2023-06-27T12:39:17.500834+0000","last_fullsized":"2023-06-27T12:39:17.500834+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14924550,"num_objects":14,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":74,"num_read_kb":27021,"num_write":78,"num_write_kb":21919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":127,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-06-27T12:39:18.474163+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:18.474163+0000","last_peered":"2023-06-27T12:39:18.474163+0000","last_clean":"2023-06-27T12:39:18.474163+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:18.474163+0000","last_undegraded":"2023-06-27T12:39:18.474163+0000","last_fullsized":"2023-06-27T12:39:18.474163+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'32","reported_seq":192,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.499914+0000","last_change":"2023-06-27T12:39:13.435356+0000","last_active":"2023-06-27T12:39:17.499914+0000","last_peered":"2023-06-27T12:39:17.499914+0000","last_clean":"2023-06-27T12:39:17.499914+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:17.499914+0000","last_undegraded":"2023-06-27T12:39:17.499914+0000","last_fullsized":"2023-06-27T12:39:17.499914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3966419,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":22,"num_read_kb":18625,"num_write":30,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.498944+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:17.498944+0000","last_peered":"2023-06-27T12:39:17.498944+0000","last_clean":"2023-06-27T12:39:17.498944+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:17.498944+0000","last_undegraded":"2023-06-27T12:39:17.498944+0000","last_fullsized":"2023-06-27T12:39:17.498944+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"80'50","reported_seq":229,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.484482+0000","last_change":"2023-06-27T12:39:13.424611+0000","last_active":"2023-06-27T12:39:17.484482+0000","last_peered":"2023-06-27T12:39:17.484482+0000","last_clean":"2023-06-27T12:39:17.484482+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:17.484482+0000","last_undegraded":"2023-06-27T12:39:17.484482+0000","last_fullsized":"2023-06-27T12:39:17.484482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10453888,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":30,"num_read_kb":27516,"num_write":46,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"82'35","reported_seq":187,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.503927+0000","last_change":"2023-06-27T12:39:14.528257+0000","last_active":"2023-06-27T12:39:17.503927+0000","last_peered":"2023-06-27T12:39:17.503927+0000","last_clean":"2023-06-27T12:39:17.503927+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:17.503927+0000","last_undegraded":"2023-06-27T12:39:17.503927+0000","last_fullsized":"2023-06-27T12:39:17.503927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5327819,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":13728,"num_write":30,"num_write_kb":8133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.485084+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:17.485084+0000","last_peered":"2023-06-27T12:39:17.485084+0000","last_clean":"2023-06-27T12:39:17.485084+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:17.485084+0000","last_undegraded":"2023-06-27T12:39:17.485084+0000","last_fullsized":"2023-06-27T12:39:17.485084+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.499781+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:17.499781+0000","last_peered":"2023-06-27T12:39:17.499781+0000","last_clean":"2023-06-27T12:39:17.499781+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:17.499781+0000","last_undegraded":"2023-06-27T12:39:17.499781+0000","last_fullsized":"2023-06-27T12:39:17.499781+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"80'43","reported_seq":220,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-06-27T12:39:17.504800+0000","last_change":"2023-06-27T12:39:13.505477+0000","last_active":"2023-06-27T12:39:17.504800+0000","last_peered":"2023-06-27T12:39:17.504800+0000","last_clean":"2023-06-27T12:39:17.504800+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:17.504800+0000","last_undegraded":"2023-06-27T12:39:17.504800+0000","last_fullsized":"2023-06-27T12:39:17.504800+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4806802,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":29,"num_read_kb":25620,"num_write":35,"num_write_kb":10153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":94236585,"num_objects":123,"num_object_clones":74,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":16,"num_read":566,"num_read_kb":363522,"num_write":636,"num_write_kb":181943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":746,"ondisk_log_size":746,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673045,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116184,"kb_used_data":89360,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94255656,"statfs":{"total":96636764160,"available":96517791744,"internally_reserved":0,"allocated":91504640,"data_stored":98647194,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673045,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108700,"kb_used_data":81876,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94263140,"statfs":{"total":96636764160,"available":96525455360,"internally_reserved":0,"allocated":83841024,"data_stored":89617478,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":104736,"kb_used_data":77912,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94267104,"statfs":{"total":96636764160,"available":96529514496,"internally_reserved":0,"allocated":79781888,"data_stored":85879655,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673045,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117304,"kb_used_data":90480,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94254536,"statfs":{"total":96636764160,"available":96516644864,"internally_reserved":0,"allocated":92651520,"data_stored":100988224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:23.544 INFO:tasks.rados.rados.0.smithi138.stdout:366: writing smithi138138488-22 from 1869389 to 2378105 tid 2 2023-06-27T12:39:23.551 INFO:tasks.rados.rados.0.smithi138.stdout:366: writing smithi138138488-22 from 3094884 to 3260558 tid 3 2023-06-27T12:39:23.551 INFO:tasks.rados.rados.0.smithi138.stdout:367: write oid 34 current snap is 40 2023-06-27T12:39:23.551 INFO:tasks.rados.rados.0.smithi138.stdout:367: seq_num 195 ranges {477365=782380,1927871=768727,3122307=155058} 2023-06-27T12:39:23.562 INFO:tasks.rados.rados.0.smithi138.stdout:367: writing smithi138138488-34 from 477365 to 1259745 tid 1 2023-06-27T12:39:23.573 INFO:tasks.rados.rados.0.smithi138.stdout:367: writing smithi138138488-34 from 1927871 to 2696598 tid 2 2023-06-27T12:39:23.577 INFO:tasks.rados.rados.0.smithi138.stdout:367: writing smithi138138488-34 from 3122307 to 3277365 tid 3 2023-06-27T12:39:23.578 INFO:tasks.rados.rados.0.smithi138.stdout:368: write oid 15 current snap is 40 2023-06-27T12:39:23.578 INFO:tasks.rados.rados.0.smithi138.stdout:365: finishing write tid 1 to smithi138138488-4 2023-06-27T12:39:23.578 INFO:tasks.rados.rados.0.smithi138.stdout:368: seq_num 196 ranges {494172=457629,1586353=512385,2749730=544442} 2023-06-27T12:39:23.585 INFO:tasks.rados.rados.0.smithi138.stdout:368: writing smithi138138488-15 from 494172 to 951801 tid 1 2023-06-27T12:39:23.594 INFO:tasks.rados.rados.0.smithi138.stdout:368: writing smithi138138488-15 from 1586353 to 2098738 tid 2 2023-06-27T12:39:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:368: writing smithi138138488-15 from 2749730 to 3294172 tid 3 2023-06-27T12:39:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:369: read oid 10 snap 38 2023-06-27T12:39:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:369: expect deleted 2023-06-27T12:39:23.604 INFO:tasks.rados.rados.0.smithi138.stdout:370: write (excl) oid 12 current snap is 40 2023-06-27T12:39:23.604 INFO:tasks.rados.rados.0.smithi138.stdout:370: seq_num 197 ranges {510979=532878,1528482=772396,3060800=250179} 2023-06-27T12:39:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:370: writing smithi138138488-12 from 510979 to 1043857 tid 1 2023-06-27T12:39:23.621 INFO:tasks.rados.rados.0.smithi138.stdout:370: writing smithi138138488-12 from 1528482 to 2300878 tid 2 2023-06-27T12:39:23.628 INFO:tasks.rados.rados.0.smithi138.stdout:370: writing smithi138138488-12 from 3060800 to 3310979 tid 3 2023-06-27T12:39:23.629 INFO:tasks.rados.rados.0.smithi138.stdout:371: read oid 37 snap 38 2023-06-27T12:39:23.629 INFO:tasks.rados.rados.0.smithi138.stdout:371: expect deleted 2023-06-27T12:39:23.629 INFO:tasks.rados.rados.0.smithi138.stdout:372: write (excl) oid 19 current snap is 40 2023-06-27T12:39:23.629 INFO:tasks.rados.rados.0.smithi138.stdout:366: finishing write tid 1 to smithi138138488-22 2023-06-27T12:39:23.629 INFO:tasks.rados.rados.0.smithi138.stdout:372: seq_num 198 ranges {527786=724480,1703317=516054,2804576=435035} 2023-06-27T12:39:23.638 INFO:tasks.rados.rados.0.smithi138.stdout:372: writing smithi138138488-19 from 527786 to 1252266 tid 1 2023-06-27T12:39:23.646 INFO:tasks.rados.rados.0.smithi138.stdout:372: writing smithi138138488-19 from 1703317 to 2219371 tid 2 2023-06-27T12:39:23.654 INFO:tasks.rados.rados.0.smithi138.stdout:372: writing smithi138138488-19 from 2804576 to 3239611 tid 3 2023-06-27T12:39:23.654 INFO:tasks.rados.rados.0.smithi138.stdout:373: snap_create 2023-06-27T12:39:23.689 INFO:tasks.rados.rados.0.smithi138.stdout:366: finishing write tid 2 to smithi138138488-22 2023-06-27T12:39:23.690 INFO:tasks.rados.rados.0.smithi138.stdout:366: finishing write tid 3 to smithi138138488-22 2023-06-27T12:39:23.694 INFO:tasks.rados.rados.0.smithi138.stdout:365: finishing write tid 2 to smithi138138488-4 2023-06-27T12:39:23.696 INFO:tasks.rados.rados.0.smithi138.stdout:366: finishing write tid 4 to smithi138138488-22 2023-06-27T12:39:23.698 INFO:tasks.rados.rados.0.smithi138.stdout:366: finishing write tid 5 to smithi138138488-22 2023-06-27T12:39:23.698 INFO:tasks.rados.rados.0.smithi138.stdout:366: oid 22 updating version 0 to 55 2023-06-27T12:39:23.698 INFO:tasks.rados.rados.0.smithi138.stdout:366: oid 22 version 55 is already newer than 55 2023-06-27T12:39:23.699 INFO:tasks.rados.rados.0.smithi138.stdout:366: oid 22 version 55 is already newer than 55 2023-06-27T12:39:23.699 INFO:tasks.rados.rados.0.smithi138.stdout:366: oid 22 version 55 is already newer than 52 2023-06-27T12:39:23.699 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 55 (ObjNum 194 snap 40 seq_num 194) dirty exists 2023-06-27T12:39:23.699 INFO:tasks.rados.rados.0.smithi138.stdout:366: left oid 22 (ObjNum 194 snap 40 seq_num 194) 2023-06-27T12:39:23.736 INFO:tasks.rados.rados.0.smithi138.stdout:367: finishing write tid 1 to smithi138138488-34 2023-06-27T12:39:23.737 INFO:tasks.rados.rados.0.smithi138.stdout:365: finishing write tid 3 to smithi138138488-4 2023-06-27T12:39:23.737 INFO:tasks.rados.rados.0.smithi138.stdout:365: finishing write tid 4 to smithi138138488-4 2023-06-27T12:39:23.737 INFO:tasks.rados.rados.0.smithi138.stdout:365: finishing write tid 5 to smithi138138488-4 2023-06-27T12:39:23.738 INFO:tasks.rados.rados.0.smithi138.stdout:365: oid 4 updating version 0 to 45 2023-06-27T12:39:23.738 INFO:tasks.rados.rados.0.smithi138.stdout:365: oid 4 version 45 is already newer than 41 2023-06-27T12:39:23.738 INFO:tasks.rados.rados.0.smithi138.stdout:365: oid 4 version 45 is already newer than 45 2023-06-27T12:39:23.739 INFO:tasks.rados.rados.0.smithi138.stdout:365: oid 4 version 45 is already newer than 45 2023-06-27T12:39:23.739 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 45 (ObjNum 193 snap 40 seq_num 193) dirty exists 2023-06-27T12:39:23.739 INFO:tasks.rados.rados.0.smithi138.stdout:365: left oid 4 (ObjNum 193 snap 40 seq_num 193) 2023-06-27T12:39:23.739 INFO:tasks.rados.rados.0.smithi138.stdout:368: finishing write tid 1 to smithi138138488-15 2023-06-27T12:39:23.745 INFO:tasks.rados.rados.0.smithi138.stdout:367: finishing write tid 2 to smithi138138488-34 2023-06-27T12:39:23.746 INFO:tasks.rados.rados.0.smithi138.stdout:367: finishing write tid 3 to smithi138138488-34 2023-06-27T12:39:23.747 INFO:tasks.rados.rados.0.smithi138.stdout:367: finishing write tid 4 to smithi138138488-34 2023-06-27T12:39:23.748 INFO:tasks.rados.rados.0.smithi138.stdout:367: finishing write tid 5 to smithi138138488-34 2023-06-27T12:39:23.748 INFO:tasks.rados.rados.0.smithi138.stdout:367: oid 34 updating version 0 to 84 2023-06-27T12:39:23.748 INFO:tasks.rados.rados.0.smithi138.stdout:367: oid 34 version 84 is already newer than 84 2023-06-27T12:39:23.748 INFO:tasks.rados.rados.0.smithi138.stdout:367: oid 34 version 84 is already newer than 84 2023-06-27T12:39:23.749 INFO:tasks.rados.rados.0.smithi138.stdout:367: oid 34 version 84 is already newer than 84 2023-06-27T12:39:23.749 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 84 (ObjNum 195 snap 40 seq_num 195) dirty exists 2023-06-27T12:39:23.749 INFO:tasks.rados.rados.0.smithi138.stdout:367: left oid 34 (ObjNum 195 snap 40 seq_num 195) 2023-06-27T12:39:23.750 INFO:tasks.rados.rados.0.smithi138.stdout:370: finishing write tid 1 to smithi138138488-12 2023-06-27T12:39:23.750 INFO:tasks.rados.rados.0.smithi138.stdout:368: finishing write tid 2 to smithi138138488-15 2023-06-27T12:39:23.751 INFO:tasks.rados.rados.0.smithi138.stdout:368: finishing write tid 3 to smithi138138488-15 2023-06-27T12:39:23.752 INFO:tasks.rados.rados.0.smithi138.stdout:368: finishing write tid 4 to smithi138138488-15 2023-06-27T12:39:23.753 INFO:tasks.rados.rados.0.smithi138.stdout:368: finishing write tid 5 to smithi138138488-15 2023-06-27T12:39:23.753 INFO:tasks.rados.rados.0.smithi138.stdout:368: oid 15 updating version 0 to 25 2023-06-27T12:39:23.753 INFO:tasks.rados.rados.0.smithi138.stdout:368: oid 15 version 25 is already newer than 25 2023-06-27T12:39:23.753 INFO:tasks.rados.rados.0.smithi138.stdout:368: oid 15 version 25 is already newer than 25 2023-06-27T12:39:23.753 INFO:tasks.rados.rados.0.smithi138.stdout:368: oid 15 version 25 is already newer than 25 2023-06-27T12:39:23.753 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 25 (ObjNum 196 snap 40 seq_num 196) dirty exists 2023-06-27T12:39:23.754 INFO:tasks.rados.rados.0.smithi138.stdout:368: left oid 15 (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:39:23.792 INFO:tasks.rados.rados.0.smithi138.stdout:370: finishing write tid 2 to smithi138138488-12 2023-06-27T12:39:23.794 INFO:tasks.rados.rados.0.smithi138.stdout:372: finishing write tid 1 to smithi138138488-19 2023-06-27T12:39:23.795 INFO:tasks.rados.rados.0.smithi138.stdout:370: finishing write tid 3 to smithi138138488-12 2023-06-27T12:39:23.811 INFO:tasks.rados.rados.0.smithi138.stdout:370: finishing write tid 4 to smithi138138488-12 2023-06-27T12:39:23.815 INFO:tasks.rados.rados.0.smithi138.stdout:370: finishing write tid 5 to smithi138138488-12 2023-06-27T12:39:23.815 INFO:tasks.rados.rados.0.smithi138.stdout:370: oid 12 updating version 0 to 67 2023-06-27T12:39:23.816 INFO:tasks.rados.rados.0.smithi138.stdout:370: oid 12 version 67 is already newer than 64 2023-06-27T12:39:23.816 INFO:tasks.rados.rados.0.smithi138.stdout:370: oid 12 version 67 is already newer than 67 2023-06-27T12:39:23.816 INFO:tasks.rados.rados.0.smithi138.stdout:370: oid 12 version 67 is already newer than 67 2023-06-27T12:39:23.816 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 67 (ObjNum 197 snap 40 seq_num 197) dirty exists 2023-06-27T12:39:23.816 INFO:tasks.rados.rados.0.smithi138.stdout:370: left oid 12 (ObjNum 197 snap 40 seq_num 197) 2023-06-27T12:39:23.818 INFO:tasks.rados.rados.0.smithi138.stdout:372: finishing write tid 2 to smithi138138488-19 2023-06-27T12:39:23.818 INFO:tasks.rados.rados.0.smithi138.stdout:372: finishing write tid 3 to smithi138138488-19 2023-06-27T12:39:23.819 INFO:tasks.rados.rados.0.smithi138.stdout:372: finishing write tid 4 to smithi138138488-19 2023-06-27T12:39:23.820 INFO:tasks.rados.rados.0.smithi138.stdout:372: finishing write tid 5 to smithi138138488-19 2023-06-27T12:39:23.820 INFO:tasks.rados.rados.0.smithi138.stdout:372: oid 19 updating version 0 to 91 2023-06-27T12:39:23.821 INFO:tasks.rados.rados.0.smithi138.stdout:372: oid 19 version 91 is already newer than 91 2023-06-27T12:39:23.821 INFO:tasks.rados.rados.0.smithi138.stdout:372: oid 19 version 91 is already newer than 91 2023-06-27T12:39:23.821 INFO:tasks.rados.rados.0.smithi138.stdout:372: oid 19 version 91 is already newer than 87 2023-06-27T12:39:23.821 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 91 (ObjNum 198 snap 40 seq_num 198) dirty exists 2023-06-27T12:39:23.821 INFO:tasks.rados.rados.0.smithi138.stdout:372: left oid 19 (ObjNum 198 snap 40 seq_num 198) 2023-06-27T12:39:24.522 INFO:tasks.rados.rados.0.smithi138.stdout:365: done (8 left) 2023-06-27T12:39:24.523 INFO:tasks.rados.rados.0.smithi138.stdout:366: done (7 left) 2023-06-27T12:39:24.523 INFO:tasks.rados.rados.0.smithi138.stdout:367: done (6 left) 2023-06-27T12:39:24.523 INFO:tasks.rados.rados.0.smithi138.stdout:368: done (5 left) 2023-06-27T12:39:24.523 INFO:tasks.rados.rados.0.smithi138.stdout:369: done (4 left) 2023-06-27T12:39:24.523 INFO:tasks.rados.rados.0.smithi138.stdout:370: done (3 left) 2023-06-27T12:39:24.524 INFO:tasks.rados.rados.0.smithi138.stdout:371: done (2 left) 2023-06-27T12:39:24.524 INFO:tasks.rados.rados.0.smithi138.stdout:372: done (1 left) 2023-06-27T12:39:24.524 INFO:tasks.rados.rados.0.smithi138.stdout:373: done (0 left) 2023-06-27T12:39:24.524 INFO:tasks.rados.rados.0.smithi138.stdout:374: write oid 33 current snap is 41 2023-06-27T12:39:24.524 INFO:tasks.rados.rados.0.smithi138.stdout:374: seq_num 199 ranges {544593=799729,1761799=776065,2948352=396241} 2023-06-27T12:39:24.529 INFO:tasks.rados.rados.0.smithi138.stdout:374: writing smithi138138488-33 from 544593 to 1344322 tid 1 2023-06-27T12:39:24.539 INFO:tasks.rados.rados.0.smithi138.stdout:374: writing smithi138138488-33 from 1761799 to 2537864 tid 2 2023-06-27T12:39:24.546 INFO:tasks.rados.rados.0.smithi138.stdout:374: writing smithi138138488-33 from 2948352 to 3344593 tid 3 2023-06-27T12:39:24.546 INFO:tasks.rados.rados.0.smithi138.stdout:375: read oid 5 snap -1 2023-06-27T12:39:24.547 INFO:tasks.rados.rados.0.smithi138.stdout:375: expect deleted 2023-06-27T12:39:24.547 INFO:tasks.rados.rados.0.smithi138.stdout:376: read oid 32 snap -1 2023-06-27T12:39:24.547 INFO:tasks.rados.rados.0.smithi138.stdout:376: expect (ObjNum 173 snap 29 seq_num 173) 2023-06-27T12:39:24.547 INFO:tasks.rados.rados.0.smithi138.stdout:377: read oid 38 snap -1 2023-06-27T12:39:24.547 INFO:tasks.rados.rados.0.smithi138.stdout:377: expect deleted 2023-06-27T12:39:24.547 INFO:tasks.rados.rados.0.smithi138.stdout:378: snap_create 2023-06-27T12:39:24.599 INFO:tasks.rados.rados.0.smithi138.stdout:376: expect (ObjNum 173 snap 29 seq_num 173) 2023-06-27T12:39:24.717 INFO:tasks.rados.rados.0.smithi138.stdout:374: finishing write tid 1 to smithi138138488-33 2023-06-27T12:39:24.717 INFO:tasks.rados.rados.0.smithi138.stdout:374: finishing write tid 2 to smithi138138488-33 2023-06-27T12:39:24.717 INFO:tasks.rados.rados.0.smithi138.stdout:374: finishing write tid 3 to smithi138138488-33 2023-06-27T12:39:24.717 INFO:tasks.rados.rados.0.smithi138.stdout:374: finishing write tid 4 to smithi138138488-33 2023-06-27T12:39:24.717 INFO:tasks.rados.rados.0.smithi138.stdout:374: finishing write tid 5 to smithi138138488-33 2023-06-27T12:39:24.718 INFO:tasks.rados.rados.0.smithi138.stdout:374: oid 33 updating version 0 to 40 2023-06-27T12:39:24.718 INFO:tasks.rados.rados.0.smithi138.stdout:374: oid 33 version 40 is already newer than 40 2023-06-27T12:39:24.718 INFO:tasks.rados.rados.0.smithi138.stdout:374: oid 33 version 40 is already newer than 40 2023-06-27T12:39:24.718 INFO:tasks.rados.rados.0.smithi138.stdout:374: oid 33 version 40 is already newer than 40 2023-06-27T12:39:24.718 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 40 (ObjNum 199 snap 41 seq_num 199) dirty exists 2023-06-27T12:39:24.718 INFO:tasks.rados.rados.0.smithi138.stdout:374: left oid 33 (ObjNum 199 snap 41 seq_num 199) 2023-06-27T12:39:24.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:24.904 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:24.899+0000 7f818ec99700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:24.907 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:24.904+0000 7f818ec99700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:25.529 INFO:tasks.rados.rados.0.smithi138.stdout:374: done (4 left) 2023-06-27T12:39:25.529 INFO:tasks.rados.rados.0.smithi138.stdout:375: done (3 left) 2023-06-27T12:39:25.529 INFO:tasks.rados.rados.0.smithi138.stdout:376: done (2 left) 2023-06-27T12:39:25.529 INFO:tasks.rados.rados.0.smithi138.stdout:377: done (1 left) 2023-06-27T12:39:25.530 INFO:tasks.rados.rados.0.smithi138.stdout:378: done (0 left) 2023-06-27T12:39:25.530 INFO:tasks.rados.rados.0.smithi138.stdout:379: delete oid 8 current snap is 42 2023-06-27T12:39:25.615 INFO:tasks.rados.rados.0.smithi138.stdout:379: done (0 left) 2023-06-27T12:39:25.615 INFO:tasks.rados.rados.0.smithi138.stdout:380: read oid 14 snap -1 2023-06-27T12:39:25.615 INFO:tasks.rados.rados.0.smithi138.stdout:380: expect (ObjNum 187 snap 36 seq_num 187) 2023-06-27T12:39:25.615 INFO:tasks.rados.rados.0.smithi138.stdout:381: snap_create 2023-06-27T12:39:25.639 INFO:tasks.rados.rados.0.smithi138.stdout:380: expect (ObjNum 187 snap 36 seq_num 187) 2023-06-27T12:39:26.536 INFO:tasks.rados.rados.0.smithi138.stdout:380: done (1 left) 2023-06-27T12:39:26.536 INFO:tasks.rados.rados.0.smithi138.stdout:381: done (0 left) 2023-06-27T12:39:26.537 INFO:tasks.rados.rados.0.smithi138.stdout:382: snap_remove snap 40 2023-06-27T12:39:26.977 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:27.116 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:27.111+0000 7fe020ee4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:27.119 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:27.116+0000 7fe020ee4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:27.539 INFO:tasks.rados.rados.0.smithi138.stdout:382: done (0 left) 2023-06-27T12:39:27.539 INFO:tasks.rados.rados.0.smithi138.stdout:383: write (excl) oid 17 current snap is 43 2023-06-27T12:39:27.539 INFO:tasks.rados.rados.0.smithi138.stdout:383: seq_num 201 ranges {578207=550227,1762410=779734,3286845=91362} 2023-06-27T12:39:27.547 INFO:tasks.rados.rados.0.smithi138.stdout:383: writing smithi138138488-17 from 578207 to 1128434 tid 1 2023-06-27T12:39:27.558 INFO:tasks.rados.rados.0.smithi138.stdout:383: writing smithi138138488-17 from 1762410 to 2542144 tid 2 2023-06-27T12:39:27.563 INFO:tasks.rados.rados.0.smithi138.stdout:383: writing smithi138138488-17 from 3286845 to 3378207 tid 3 2023-06-27T12:39:27.564 INFO:tasks.rados.rados.0.smithi138.stdout:384: read oid 20 snap -1 2023-06-27T12:39:27.564 INFO:tasks.rados.rados.0.smithi138.stdout:384: expect (ObjNum 153 snap 21 seq_num 153) 2023-06-27T12:39:27.564 INFO:tasks.rados.rados.0.smithi138.stdout:385: delete oid 42 current snap is 43 2023-06-27T12:39:27.601 INFO:tasks.rados.rados.0.smithi138.stdout:385: done (2 left) 2023-06-27T12:39:27.601 INFO:tasks.rados.rados.0.smithi138.stdout:386: read oid 12 snap 41 2023-06-27T12:39:27.601 INFO:tasks.rados.rados.0.smithi138.stdout:386: expect (ObjNum 197 snap 40 seq_num 197) 2023-06-27T12:39:27.602 INFO:tasks.rados.rados.0.smithi138.stdout:387: read oid 46 snap -1 2023-06-27T12:39:27.602 INFO:tasks.rados.rados.0.smithi138.stdout:387: expect deleted 2023-06-27T12:39:27.602 INFO:tasks.rados.rados.0.smithi138.stdout:388: read oid 47 snap -1 2023-06-27T12:39:27.602 INFO:tasks.rados.rados.0.smithi138.stdout:388: expect (ObjNum 192 snap 38 seq_num 192) 2023-06-27T12:39:27.603 INFO:tasks.rados.rados.0.smithi138.stdout:389: read oid 26 snap -1 2023-06-27T12:39:27.603 INFO:tasks.rados.rados.0.smithi138.stdout:389: expect (ObjNum 180 snap 33 seq_num 180) 2023-06-27T12:39:27.603 INFO:tasks.rados.rados.0.smithi138.stdout:390: read oid 43 snap -1 2023-06-27T12:39:27.603 INFO:tasks.rados.rados.0.smithi138.stdout:390: expect deleted 2023-06-27T12:39:27.603 INFO:tasks.rados.rados.0.smithi138.stdout:391: snap_create 2023-06-27T12:39:27.633 INFO:tasks.rados.rados.0.smithi138.stdout:383: finishing write tid 1 to smithi138138488-17 2023-06-27T12:39:27.646 INFO:tasks.rados.rados.0.smithi138.stdout:386: expect (ObjNum 197 snap 40 seq_num 197) 2023-06-27T12:39:27.771 INFO:tasks.rados.rados.0.smithi138.stdout:389: expect (ObjNum 180 snap 33 seq_num 180) 2023-06-27T12:39:27.887 INFO:tasks.rados.rados.0.smithi138.stdout:383: finishing write tid 2 to smithi138138488-17 2023-06-27T12:39:27.888 INFO:tasks.rados.rados.0.smithi138.stdout:383: finishing write tid 3 to smithi138138488-17 2023-06-27T12:39:27.888 INFO:tasks.rados.rados.0.smithi138.stdout:383: finishing write tid 4 to smithi138138488-17 2023-06-27T12:39:27.888 INFO:tasks.rados.rados.0.smithi138.stdout:383: finishing write tid 5 to smithi138138488-17 2023-06-27T12:39:27.888 INFO:tasks.rados.rados.0.smithi138.stdout:383: oid 17 updating version 0 to 73 2023-06-27T12:39:27.888 INFO:tasks.rados.rados.0.smithi138.stdout:383: oid 17 version 73 is already newer than 69 2023-06-27T12:39:27.888 INFO:tasks.rados.rados.0.smithi138.stdout:383: oid 17 version 73 is already newer than 73 2023-06-27T12:39:27.888 INFO:tasks.rados.rados.0.smithi138.stdout:383: oid 17 version 73 is already newer than 73 2023-06-27T12:39:27.889 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 73 (ObjNum 201 snap 43 seq_num 201) dirty exists 2023-06-27T12:39:27.889 INFO:tasks.rados.rados.0.smithi138.stdout:383: left oid 17 (ObjNum 201 snap 43 seq_num 201) 2023-06-27T12:39:27.889 INFO:tasks.rados.rados.0.smithi138.stdout:384: expect (ObjNum 153 snap 21 seq_num 153) 2023-06-27T12:39:28.017 INFO:tasks.rados.rados.0.smithi138.stdout:388: expect (ObjNum 192 snap 38 seq_num 192) 2023-06-27T12:39:28.542 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:28.542 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:28.543 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:28.563 INFO:tasks.rados.rados.0.smithi138.stdout:383: done (7 left) 2023-06-27T12:39:28.564 INFO:tasks.rados.rados.0.smithi138.stdout:384: done (6 left) 2023-06-27T12:39:28.564 INFO:tasks.rados.rados.0.smithi138.stdout:386: done (5 left) 2023-06-27T12:39:28.564 INFO:tasks.rados.rados.0.smithi138.stdout:387: done (4 left) 2023-06-27T12:39:28.564 INFO:tasks.rados.rados.0.smithi138.stdout:388: done (3 left) 2023-06-27T12:39:28.564 INFO:tasks.rados.rados.0.smithi138.stdout:389: done (2 left) 2023-06-27T12:39:28.565 INFO:tasks.rados.rados.0.smithi138.stdout:390: done (1 left) 2023-06-27T12:39:28.565 INFO:tasks.rados.rados.0.smithi138.stdout:391: done (0 left) 2023-06-27T12:39:28.565 INFO:tasks.rados.rados.0.smithi138.stdout:392: read oid 26 snap 37 2023-06-27T12:39:28.565 INFO:tasks.rados.rados.0.smithi138.stdout:392: expect (ObjNum 180 snap 33 seq_num 180) 2023-06-27T12:39:28.565 INFO:tasks.rados.rados.0.smithi138.stdout:393: delete oid 29 current snap is 44 2023-06-27T12:39:28.611 INFO:tasks.rados.rados.0.smithi138.stdout:392: expect (ObjNum 180 snap 33 seq_num 180) 2023-06-27T12:39:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:28.689+0000 7fa0e4849700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:28.699 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:28.695+0000 7fa0e4849700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:28.751 INFO:tasks.rados.rados.0.smithi138.stdout:392: done (1 left) 2023-06-27T12:39:28.751 INFO:tasks.rados.rados.0.smithi138.stdout:393: done (0 left) 2023-06-27T12:39:28.751 INFO:tasks.rados.rados.0.smithi138.stdout:394: write oid 4 current snap is 44 2023-06-27T12:39:28.751 INFO:tasks.rados.rados.0.smithi138.stdout:394: seq_num 204 ranges {628628=775974,1937856=527061,3252761=175867} 2023-06-27T12:39:28.758 INFO:tasks.rados.rados.0.smithi138.stdout:394: writing smithi138138488-4 from 628628 to 1404602 tid 1 2023-06-27T12:39:28.764 INFO:tasks.rados.rados.0.smithi138.stdout:394: writing smithi138138488-4 from 1937856 to 2464917 tid 2 2023-06-27T12:39:28.771 INFO:tasks.rados.rados.0.smithi138.stdout:394: writing smithi138138488-4 from 3252761 to 3428628 tid 3 2023-06-27T12:39:28.771 INFO:tasks.rados.rados.0.smithi138.stdout:395: read oid 29 snap -1 2023-06-27T12:39:28.771 INFO:tasks.rados.rados.0.smithi138.stdout:395: expect deleted 2023-06-27T12:39:28.772 INFO:tasks.rados.rados.0.smithi138.stdout:396: read oid 44 snap -1 2023-06-27T12:39:28.772 INFO:tasks.rados.rados.0.smithi138.stdout:396: expect (ObjNum 105 snap 9 seq_num 105) 2023-06-27T12:39:28.772 INFO:tasks.rados.rados.0.smithi138.stdout:397: write (excl) oid 23 current snap is 44 2023-06-27T12:39:28.773 INFO:tasks.rados.rados.0.smithi138.stdout:397: seq_num 205 ranges {645435=451223,1879985=787072,3280184=165251} 2023-06-27T12:39:28.779 INFO:tasks.rados.rados.0.smithi138.stdout:397: writing smithi138138488-23 from 645435 to 1096658 tid 1 2023-06-27T12:39:28.791 INFO:tasks.rados.rados.0.smithi138.stdout:397: writing smithi138138488-23 from 1879985 to 2667057 tid 2 2023-06-27T12:39:28.797 INFO:tasks.rados.rados.0.smithi138.stdout:397: writing smithi138138488-23 from 3280184 to 3445435 tid 3 2023-06-27T12:39:28.797 INFO:tasks.rados.rados.0.smithi138.stdout:398: delete oid 31 current snap is 44 2023-06-27T12:39:28.822 INFO:tasks.rados.rados.0.smithi138.stdout:394: finishing write tid 1 to smithi138138488-4 2023-06-27T12:39:28.830 INFO:tasks.rados.rados.0.smithi138.stdout:396: expect (ObjNum 105 snap 9 seq_num 105) 2023-06-27T12:39:28.925 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:28.927 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:28.940 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2023-06-27T12:39:28.556723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":106743215,"num_objects":133,"num_object_clones":80,"num_object_copies":399,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":15,"num_read":701,"num_read_kb":387572,"num_write":818,"num_write_kb":197582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":910,"ondisk_log_size":910,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":487088,"kb_used_data":375696,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":377000272,"statfs":{"total":386547056640,"available":386048278528,"internally_reserved":0,"allocated":384712704,"data_stored":420310219,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040303"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":174,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.598328+0000","last_change":"2023-06-27T12:39:22.509209+0000","last_active":"2023-06-27T12:39:23.598328+0000","last_peered":"2023-06-27T12:39:23.598328+0000","last_clean":"2023-06-27T12:39:23.598328+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:23.598328+0000","last_undegraded":"2023-06-27T12:39:23.598328+0000","last_fullsized":"2023-06-27T12:39:23.598328+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"92'66","reported_seq":261,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.558013+0000","last_change":"2023-06-27T12:39:22.893614+0000","last_active":"2023-06-27T12:39:23.558013+0000","last_peered":"2023-06-27T12:39:23.558013+0000","last_clean":"2023-06-27T12:39:23.558013+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:23.558013+0000","last_undegraded":"2023-06-27T12:39:23.558013+0000","last_fullsized":"2023-06-27T12:39:23.558013+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8907710,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":39,"num_read_kb":28514,"num_write":58,"num_write_kb":16873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"95'67","reported_seq":249,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.777863+0000","last_change":"2023-06-27T12:39:22.831694+0000","last_active":"2023-06-27T12:39:23.777863+0000","last_peered":"2023-06-27T12:39:23.777863+0000","last_clean":"2023-06-27T12:39:23.777863+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:23.777863+0000","last_undegraded":"2023-06-27T12:39:23.777863+0000","last_fullsized":"2023-06-27T12:39:23.777863+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9613604,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":37,"num_read_kb":12458,"num_write":61,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"48'34","reported_seq":215,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.609631+0000","last_change":"2023-06-27T12:39:22.524870+0000","last_active":"2023-06-27T12:39:23.609631+0000","last_peered":"2023-06-27T12:39:23.609631+0000","last_clean":"2023-06-27T12:39:23.609631+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:23.609631+0000","last_undegraded":"2023-06-27T12:39:23.609631+0000","last_fullsized":"2023-06-27T12:39:23.609631+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4134522,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":37,"num_read_kb":13465,"num_write":30,"num_write_kb":8404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"95'25","reported_seq":197,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.632818+0000","last_change":"2023-06-27T12:39:22.900379+0000","last_active":"2023-06-27T12:39:23.632818+0000","last_peered":"2023-06-27T12:39:23.632818+0000","last_clean":"2023-06-27T12:39:23.632818+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:23.632818+0000","last_undegraded":"2023-06-27T12:39:23.632818+0000","last_fullsized":"2023-06-27T12:39:23.632818+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5311012,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":14491,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"95'84","reported_seq":280,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.683040+0000","last_change":"2023-06-27T12:39:22.527193+0000","last_active":"2023-06-27T12:39:23.683040+0000","last_peered":"2023-06-27T12:39:23.683040+0000","last_clean":"2023-06-27T12:39:23.683040+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:23.683040+0000","last_undegraded":"2023-06-27T12:39:23.683040+0000","last_fullsized":"2023-06-27T12:39:23.683040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16033845,"num_objects":15,"num_object_clones":9,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":105,"num_read_kb":66648,"num_write":72,"num_write_kb":20414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":245,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.628865+0000","last_change":"2023-06-27T12:39:22.524974+0000","last_active":"2023-06-27T12:39:23.628865+0000","last_peered":"2023-06-27T12:39:23.628865+0000","last_clean":"2023-06-27T12:39:23.628865+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:23.628865+0000","last_undegraded":"2023-06-27T12:39:23.628865+0000","last_fullsized":"2023-06-27T12:39:23.628865+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":24452,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"65'14","reported_seq":200,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.671507+0000","last_change":"2023-06-27T12:39:22.829718+0000","last_active":"2023-06-27T12:39:23.671507+0000","last_peered":"2023-06-27T12:39:23.671507+0000","last_clean":"2023-06-27T12:39:23.671507+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:23.671507+0000","last_undegraded":"2023-06-27T12:39:23.671507+0000","last_fullsized":"2023-06-27T12:39:23.671507+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2705927,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":25158,"num_write":13,"num_write_kb":3961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":147,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.555728+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:23.555728+0000","last_peered":"2023-06-27T12:39:23.555728+0000","last_clean":"2023-06-27T12:39:23.555728+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:23.555728+0000","last_undegraded":"2023-06-27T12:39:23.555728+0000","last_fullsized":"2023-06-27T12:39:23.555728+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":173,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.666762+0000","last_change":"2023-06-27T12:39:22.832135+0000","last_active":"2023-06-27T12:39:23.666762+0000","last_peered":"2023-06-27T12:39:23.666762+0000","last_clean":"2023-06-27T12:39:23.666762+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:23.666762+0000","last_undegraded":"2023-06-27T12:39:23.666762+0000","last_fullsized":"2023-06-27T12:39:23.666762+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.597502+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:23.597502+0000","last_peered":"2023-06-27T12:39:23.597502+0000","last_clean":"2023-06-27T12:39:23.597502+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:23.597502+0000","last_undegraded":"2023-06-27T12:39:23.597502+0000","last_fullsized":"2023-06-27T12:39:23.597502+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"69'41","reported_seq":228,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.556504+0000","last_change":"2023-06-27T12:39:22.844413+0000","last_active":"2023-06-27T12:39:23.556504+0000","last_peered":"2023-06-27T12:39:23.556504+0000","last_clean":"2023-06-27T12:39:23.556504+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:23.556504+0000","last_undegraded":"2023-06-27T12:39:23.556504+0000","last_fullsized":"2023-06-27T12:39:23.556504+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722734,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":20336,"num_write":33,"num_write_kb":9145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.674769+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:23.674769+0000","last_peered":"2023-06-27T12:39:23.674769+0000","last_clean":"2023-06-27T12:39:23.674769+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:23.674769+0000","last_undegraded":"2023-06-27T12:39:23.674769+0000","last_fullsized":"2023-06-27T12:39:23.674769+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"88'125","reported_seq":288,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.512221+0000","last_change":"2023-06-27T12:39:23.512221+0000","last_active":"2023-06-27T12:39:23.512221+0000","last_peered":"2023-06-27T12:39:23.512221+0000","last_clean":"2023-06-27T12:39:23.512221+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:23.512221+0000","last_undegraded":"2023-06-27T12:39:23.512221+0000","last_fullsized":"2023-06-27T12:39:23.512221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8605184,"num_objects":16,"num_object_clones":11,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":66,"num_read_kb":57048,"num_write":95,"num_write_kb":27990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":141,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.664745+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:23.664745+0000","last_peered":"2023-06-27T12:39:23.664745+0000","last_clean":"2023-06-27T12:39:23.664745+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:23.664745+0000","last_undegraded":"2023-06-27T12:39:23.664745+0000","last_fullsized":"2023-06-27T12:39:23.664745+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":260,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.508170+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:23.508170+0000","last_peered":"2023-06-27T12:39:23.508170+0000","last_clean":"2023-06-27T12:39:23.508170+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:23.508170+0000","last_undegraded":"2023-06-27T12:39:23.508170+0000","last_fullsized":"2023-06-27T12:39:23.508170+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"95'91","reported_seq":288,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.797255+0000","last_change":"2023-06-27T12:39:22.830141+0000","last_active":"2023-06-27T12:39:23.797255+0000","last_peered":"2023-06-27T12:39:23.797255+0000","last_clean":"2023-06-27T12:39:23.797255+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:23.797255+0000","last_undegraded":"2023-06-27T12:39:23.797255+0000","last_fullsized":"2023-06-27T12:39:23.797255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15176655,"num_objects":15,"num_object_clones":9,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":83,"num_read_kb":33374,"num_write":83,"num_write_kb":23556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.599066+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:23.599066+0000","last_peered":"2023-06-27T12:39:23.599066+0000","last_clean":"2023-06-27T12:39:23.599066+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:23.599066+0000","last_undegraded":"2023-06-27T12:39:23.599066+0000","last_fullsized":"2023-06-27T12:39:23.599066+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"95'45","reported_seq":211,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.578001+0000","last_change":"2023-06-27T12:39:22.815268+0000","last_active":"2023-06-27T12:39:23.578001+0000","last_peered":"2023-06-27T12:39:23.578001+0000","last_clean":"2023-06-27T12:39:23.578001+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:23.578001+0000","last_undegraded":"2023-06-27T12:39:23.578001+0000","last_fullsized":"2023-06-27T12:39:23.578001+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3243751,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":26,"num_read_kb":24671,"num_write":35,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.505018+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:23.505018+0000","last_peered":"2023-06-27T12:39:23.505018+0000","last_clean":"2023-06-27T12:39:23.505018+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:23.505018+0000","last_undegraded":"2023-06-27T12:39:23.505018+0000","last_fullsized":"2023-06-27T12:39:23.505018+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"80'50","reported_seq":241,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.560164+0000","last_change":"2023-06-27T12:39:22.838923+0000","last_active":"2023-06-27T12:39:23.560164+0000","last_peered":"2023-06-27T12:39:23.560164+0000","last_clean":"2023-06-27T12:39:23.560164+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:23.560164+0000","last_undegraded":"2023-06-27T12:39:23.560164+0000","last_fullsized":"2023-06-27T12:39:23.560164+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10453888,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":30,"num_read_kb":27516,"num_write":46,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"82'35","reported_seq":200,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.507845+0000","last_change":"2023-06-27T12:39:22.814955+0000","last_active":"2023-06-27T12:39:23.507845+0000","last_peered":"2023-06-27T12:39:23.507845+0000","last_clean":"2023-06-27T12:39:23.507845+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:23.507845+0000","last_undegraded":"2023-06-27T12:39:23.507845+0000","last_fullsized":"2023-06-27T12:39:23.507845+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5327819,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":13728,"num_write":30,"num_write_kb":8133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":147,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.555918+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:23.555918+0000","last_peered":"2023-06-27T12:39:23.555918+0000","last_clean":"2023-06-27T12:39:23.555918+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:23.555918+0000","last_undegraded":"2023-06-27T12:39:23.555918+0000","last_fullsized":"2023-06-27T12:39:23.555918+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.669765+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:23.669765+0000","last_peered":"2023-06-27T12:39:23.669765+0000","last_clean":"2023-06-27T12:39:23.669765+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:23.669765+0000","last_undegraded":"2023-06-27T12:39:23.669765+0000","last_fullsized":"2023-06-27T12:39:23.669765+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"95'55","reported_seq":237,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-06-27T12:39:23.617186+0000","last_change":"2023-06-27T12:39:22.834573+0000","last_active":"2023-06-27T12:39:23.617186+0000","last_peered":"2023-06-27T12:39:23.617186+0000","last_clean":"2023-06-27T12:39:23.617186+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:23.617186+0000","last_undegraded":"2023-06-27T12:39:23.617186+0000","last_fullsized":"2023-06-27T12:39:23.617186+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8067360,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":25621,"num_write":40,"num_write_kb":11503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":106152847,"num_objects":131,"num_object_clones":80,"num_object_copies":393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":15,"num_read":598,"num_read_kb":387481,"num_write":681,"num_write_kb":195688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":813,"ondisk_log_size":813,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":129236,"kb_used_data":98316,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94242604,"statfs":{"total":96636764160,"available":96504426496,"internally_reserved":0,"allocated":100675584,"data_stored":109453208,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":114304,"kb_used_data":87480,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94257536,"statfs":{"total":96636764160,"available":96519716864,"internally_reserved":0,"allocated":89579520,"data_stored":97352592,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705755,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115720,"kb_used_data":88896,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94256120,"statfs":{"total":96636764160,"available":96518266880,"internally_reserved":0,"allocated":91029504,"data_stored":98976998,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":127828,"kb_used_data":101004,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94244012,"statfs":{"total":96636764160,"available":96505868288,"internally_reserved":0,"allocated":103428096,"data_stored":114527421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:29.066 INFO:tasks.rados.rados.0.smithi138.stdout:397: finishing write tid 1 to smithi138138488-23 2023-06-27T12:39:29.066 INFO:tasks.rados.rados.0.smithi138.stdout:394: finishing write tid 2 to smithi138138488-4 2023-06-27T12:39:29.067 INFO:tasks.rados.rados.0.smithi138.stdout:394: finishing write tid 3 to smithi138138488-4 2023-06-27T12:39:29.067 INFO:tasks.rados.rados.0.smithi138.stdout:394: finishing write tid 4 to smithi138138488-4 2023-06-27T12:39:29.067 INFO:tasks.rados.rados.0.smithi138.stdout:394: finishing write tid 5 to smithi138138488-4 2023-06-27T12:39:29.067 INFO:tasks.rados.rados.0.smithi138.stdout:394: oid 4 updating version 0 to 50 2023-06-27T12:39:29.067 INFO:tasks.rados.rados.0.smithi138.stdout:394: oid 4 version 50 is already newer than 50 2023-06-27T12:39:29.067 INFO:tasks.rados.rados.0.smithi138.stdout:394: oid 4 version 50 is already newer than 50 2023-06-27T12:39:29.067 INFO:tasks.rados.rados.0.smithi138.stdout:394: oid 4 version 50 is already newer than 50 2023-06-27T12:39:29.068 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 50 (ObjNum 204 snap 44 seq_num 204) dirty exists 2023-06-27T12:39:29.068 INFO:tasks.rados.rados.0.smithi138.stdout:394: left oid 4 (ObjNum 204 snap 44 seq_num 204) 2023-06-27T12:39:29.068 INFO:tasks.rados.rados.0.smithi138.stdout:397: finishing write tid 2 to smithi138138488-23 2023-06-27T12:39:29.068 INFO:tasks.rados.rados.0.smithi138.stdout:394: done (4 left) 2023-06-27T12:39:29.068 INFO:tasks.rados.rados.0.smithi138.stdout:395: done (3 left) 2023-06-27T12:39:29.068 INFO:tasks.rados.rados.0.smithi138.stdout:396: done (2 left) 2023-06-27T12:39:29.068 INFO:tasks.rados.rados.0.smithi138.stdout:398: done (1 left) 2023-06-27T12:39:29.069 INFO:tasks.rados.rados.0.smithi138.stdout:399: write oid 50 current snap is 44 2023-06-27T12:39:29.069 INFO:tasks.rados.rados.0.smithi138.stdout:397: finishing write tid 3 to smithi138138488-23 2023-06-27T12:39:29.069 INFO:tasks.rados.rados.0.smithi138.stdout:397: finishing write tid 4 to smithi138138488-23 2023-06-27T12:39:29.069 INFO:tasks.rados.rados.0.smithi138.stdout:397: finishing write tid 5 to smithi138138488-23 2023-06-27T12:39:29.069 INFO:tasks.rados.rados.0.smithi138.stdout:397: oid 23 updating version 0 to 51 2023-06-27T12:39:29.069 INFO:tasks.rados.rados.0.smithi138.stdout:397: oid 23 version 51 is already newer than 51 2023-06-27T12:39:29.069 INFO:tasks.rados.rados.0.smithi138.stdout:397: oid 23 version 51 is already newer than 51 2023-06-27T12:39:29.069 INFO:tasks.rados.rados.0.smithi138.stdout:397: oid 23 version 51 is already newer than 48 2023-06-27T12:39:29.070 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 51 (ObjNum 205 snap 44 seq_num 205) dirty exists 2023-06-27T12:39:29.070 INFO:tasks.rados.rados.0.smithi138.stdout:397: left oid 23 (ObjNum 205 snap 44 seq_num 205) 2023-06-27T12:39:29.070 INFO:tasks.rados.rados.0.smithi138.stdout:399: seq_num 207 ranges {679049=718074,2113302=674388,3335030=144019} 2023-06-27T12:39:29.073 INFO:tasks.rados.rados.0.smithi138.stdout:399: writing smithi138138488-50 from 679049 to 1397123 tid 1 2023-06-27T12:39:29.083 INFO:tasks.rados.rados.0.smithi138.stdout:399: writing smithi138138488-50 from 2113302 to 2787690 tid 2 2023-06-27T12:39:29.088 INFO:tasks.rados.rados.0.smithi138.stdout:399: writing smithi138138488-50 from 3335030 to 3479049 tid 3 2023-06-27T12:39:29.088 INFO:tasks.rados.rados.0.smithi138.stdout:397: done (1 left) 2023-06-27T12:39:29.088 INFO:tasks.rados.rados.0.smithi138.stdout:400: write oid 24 current snap is 44 2023-06-27T12:39:29.089 INFO:tasks.rados.rados.0.smithi138.stdout:400: seq_num 208 ranges {695856=793323,2171784=534399,3362453=133403} 2023-06-27T12:39:29.100 INFO:tasks.rados.rados.0.smithi138.stdout:400: writing smithi138138488-24 from 695856 to 1489179 tid 1 2023-06-27T12:39:29.110 INFO:tasks.rados.rados.0.smithi138.stdout:400: writing smithi138138488-24 from 2171784 to 2706183 tid 2 2023-06-27T12:39:29.115 INFO:tasks.rados.rados.0.smithi138.stdout:400: writing smithi138138488-24 from 3362453 to 3495856 tid 3 2023-06-27T12:39:29.115 INFO:tasks.rados.rados.0.smithi138.stdout:401: read oid 15 snap -1 2023-06-27T12:39:29.116 INFO:tasks.rados.rados.0.smithi138.stdout:401: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:39:29.116 INFO:tasks.rados.rados.0.smithi138.stdout:402: snap_create 2023-06-27T12:39:29.130 INFO:tasks.rados.rados.0.smithi138.stdout:399: finishing write tid 1 to smithi138138488-50 2023-06-27T12:39:29.147 INFO:tasks.rados.rados.0.smithi138.stdout:399: finishing write tid 2 to smithi138138488-50 2023-06-27T12:39:29.155 INFO:tasks.rados.rados.0.smithi138.stdout:401: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:39:29.192 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:29.287 INFO:tasks.rados.rados.0.smithi138.stdout:399: finishing write tid 3 to smithi138138488-50 2023-06-27T12:39:29.288 INFO:tasks.rados.rados.0.smithi138.stdout:400: finishing write tid 1 to smithi138138488-24 2023-06-27T12:39:29.288 INFO:tasks.rados.rados.0.smithi138.stdout:400: finishing write tid 2 to smithi138138488-24 2023-06-27T12:39:29.288 INFO:tasks.rados.rados.0.smithi138.stdout:400: finishing write tid 3 to smithi138138488-24 2023-06-27T12:39:29.288 INFO:tasks.rados.rados.0.smithi138.stdout:400: finishing write tid 4 to smithi138138488-24 2023-06-27T12:39:29.288 INFO:tasks.rados.rados.0.smithi138.stdout:399: finishing write tid 4 to smithi138138488-50 2023-06-27T12:39:29.289 INFO:tasks.rados.rados.0.smithi138.stdout:399: finishing write tid 5 to smithi138138488-50 2023-06-27T12:39:29.289 INFO:tasks.rados.rados.0.smithi138.stdout:399: oid 50 updating version 0 to 23 2023-06-27T12:39:29.289 INFO:tasks.rados.rados.0.smithi138.stdout:399: oid 50 version 23 is already newer than 23 2023-06-27T12:39:29.289 INFO:tasks.rados.rados.0.smithi138.stdout:399: oid 50 version 23 is already newer than 23 2023-06-27T12:39:29.289 INFO:tasks.rados.rados.0.smithi138.stdout:399: oid 50 version 23 is already newer than 23 2023-06-27T12:39:29.289 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 23 (ObjNum 207 snap 44 seq_num 207) dirty exists 2023-06-27T12:39:29.289 INFO:tasks.rados.rados.0.smithi138.stdout:399: left oid 50 (ObjNum 207 snap 44 seq_num 207) 2023-06-27T12:39:29.289 INFO:tasks.rados.rados.0.smithi138.stdout:400: finishing write tid 5 to smithi138138488-24 2023-06-27T12:39:29.290 INFO:tasks.rados.rados.0.smithi138.stdout:400: oid 24 updating version 0 to 91 2023-06-27T12:39:29.290 INFO:tasks.rados.rados.0.smithi138.stdout:400: oid 24 version 91 is already newer than 91 2023-06-27T12:39:29.290 INFO:tasks.rados.rados.0.smithi138.stdout:400: oid 24 version 91 is already newer than 91 2023-06-27T12:39:29.290 INFO:tasks.rados.rados.0.smithi138.stdout:400: oid 24 version 91 is already newer than 91 2023-06-27T12:39:29.290 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 91 (ObjNum 208 snap 44 seq_num 208) dirty exists 2023-06-27T12:39:29.290 INFO:tasks.rados.rados.0.smithi138.stdout:400: left oid 24 (ObjNum 208 snap 44 seq_num 208) 2023-06-27T12:39:29.320 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:29.315+0000 7fa2e292f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:29.322 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:29.319+0000 7fa2e292f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:29.567 INFO:tasks.rados.rados.0.smithi138.stdout:399: done (3 left) 2023-06-27T12:39:29.568 INFO:tasks.rados.rados.0.smithi138.stdout:400: done (2 left) 2023-06-27T12:39:29.568 INFO:tasks.rados.rados.0.smithi138.stdout:401: done (1 left) 2023-06-27T12:39:29.568 INFO:tasks.rados.rados.0.smithi138.stdout:402: done (0 left) 2023-06-27T12:39:29.568 INFO:tasks.rados.rados.0.smithi138.stdout:403: read oid 17 snap -1 2023-06-27T12:39:29.568 INFO:tasks.rados.rados.0.smithi138.stdout:403: expect (ObjNum 201 snap 43 seq_num 201) 2023-06-27T12:39:29.568 INFO:tasks.rados.rados.0.smithi138.stdout:404: read oid 35 snap 37 2023-06-27T12:39:29.568 INFO:tasks.rados.rados.0.smithi138.stdout:404: expect deleted 2023-06-27T12:39:29.569 INFO:tasks.rados.rados.0.smithi138.stdout:405: read oid 8 snap -1 2023-06-27T12:39:29.569 INFO:tasks.rados.rados.0.smithi138.stdout:405: expect deleted 2023-06-27T12:39:29.569 INFO:tasks.rados.rados.0.smithi138.stdout:406: snap_remove snap 44 2023-06-27T12:39:30.388 INFO:tasks.rados.rados.0.smithi138.stdout:406: done (3 left) 2023-06-27T12:39:30.389 INFO:tasks.rados.rados.0.smithi138.stdout:407: delete oid 4 current snap is 45 2023-06-27T12:39:30.389 INFO:tasks.rados.rados.0.smithi138.stdout:403: expect (ObjNum 201 snap 43 seq_num 201) 2023-06-27T12:39:30.611 INFO:tasks.rados.rados.0.smithi138.stdout:403: done (3 left) 2023-06-27T12:39:30.611 INFO:tasks.rados.rados.0.smithi138.stdout:404: done (2 left) 2023-06-27T12:39:30.611 INFO:tasks.rados.rados.0.smithi138.stdout:405: done (1 left) 2023-06-27T12:39:30.611 INFO:tasks.rados.rados.0.smithi138.stdout:407: done (0 left) 2023-06-27T12:39:30.611 INFO:tasks.rados.rados.0.smithi138.stdout:408: write oid 13 current snap is 45 2023-06-27T12:39:30.611 INFO:tasks.rados.rados.0.smithi138.stdout:408: seq_num 210 ranges {729470=543821,1772395=538068,2900946=403005} 2023-06-27T12:39:30.617 INFO:tasks.rados.rados.0.smithi138.stdout:408: writing smithi138138488-13 from 729470 to 1273291 tid 1 2023-06-27T12:39:30.622 INFO:tasks.rados.rados.0.smithi138.stdout:408: writing smithi138138488-13 from 1772395 to 2310463 tid 2 2023-06-27T12:39:30.628 INFO:tasks.rados.rados.0.smithi138.stdout:408: writing smithi138138488-13 from 2900946 to 3303951 tid 3 2023-06-27T12:39:30.628 INFO:tasks.rados.rados.0.smithi138.stdout:409: write (excl) oid 14 current snap is 45 2023-06-27T12:39:30.628 INFO:tasks.rados.rados.0.smithi138.stdout:409: seq_num 211 ranges {746277=619070,1830877=681726,2928369=617908} 2023-06-27T12:39:30.635 INFO:tasks.rados.rados.0.smithi138.stdout:409: writing smithi138138488-14 from 746277 to 1365347 tid 1 2023-06-27T12:39:30.645 INFO:tasks.rados.rados.0.smithi138.stdout:409: writing smithi138138488-14 from 1830877 to 2512603 tid 2 2023-06-27T12:39:30.654 INFO:tasks.rados.rados.0.smithi138.stdout:409: writing smithi138138488-14 from 2928369 to 3546277 tid 3 2023-06-27T12:39:30.654 INFO:tasks.rados.rados.0.smithi138.stdout:410: read oid 27 snap -1 2023-06-27T12:39:30.654 INFO:tasks.rados.rados.0.smithi138.stdout:410: expect deleted 2023-06-27T12:39:30.654 INFO:tasks.rados.rados.0.smithi138.stdout:411: snap_remove snap 37 2023-06-27T12:39:31.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:31.523 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:31.518+0000 7efc1a542700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:31.522+0000 7efc1a542700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:31.583 INFO:tasks.rados.rados.0.smithi138.stdout:411: done (3 left) 2023-06-27T12:39:31.583 INFO:tasks.rados.rados.0.smithi138.stdout:412: write (excl) oid 31 current snap is 45 2023-06-27T12:39:31.583 INFO:tasks.rados.rados.0.smithi138.stdout:408: finishing write tid 1 to smithi138138488-13 2023-06-27T12:39:31.584 INFO:tasks.rados.rados.0.smithi138.stdout:412: seq_num 212 ranges {763084=694319,1889359=541737,2955792=541902} 2023-06-27T12:39:31.591 INFO:tasks.rados.rados.0.smithi138.stdout:412: writing smithi138138488-31 from 763084 to 1457403 tid 1 2023-06-27T12:39:31.599 INFO:tasks.rados.rados.0.smithi138.stdout:412: writing smithi138138488-31 from 1889359 to 2431096 tid 2 2023-06-27T12:39:31.608 INFO:tasks.rados.rados.0.smithi138.stdout:412: writing smithi138138488-31 from 2955792 to 3497694 tid 3 2023-06-27T12:39:31.608 INFO:tasks.rados.rados.0.smithi138.stdout:413: snap_create 2023-06-27T12:39:31.608 INFO:tasks.rados.rados.0.smithi138.stdout:408: finishing write tid 2 to smithi138138488-13 2023-06-27T12:39:31.608 INFO:tasks.rados.rados.0.smithi138.stdout:409: finishing write tid 1 to smithi138138488-14 2023-06-27T12:39:31.608 INFO:tasks.rados.rados.0.smithi138.stdout:409: finishing write tid 2 to smithi138138488-14 2023-06-27T12:39:31.608 INFO:tasks.rados.rados.0.smithi138.stdout:409: finishing write tid 3 to smithi138138488-14 2023-06-27T12:39:31.609 INFO:tasks.rados.rados.0.smithi138.stdout:409: finishing write tid 4 to smithi138138488-14 2023-06-27T12:39:31.609 INFO:tasks.rados.rados.0.smithi138.stdout:409: finishing write tid 5 to smithi138138488-14 2023-06-27T12:39:31.609 INFO:tasks.rados.rados.0.smithi138.stdout:409: oid 14 updating version 0 to 92 2023-06-27T12:39:31.609 INFO:tasks.rados.rados.0.smithi138.stdout:409: oid 14 updating version 92 to 96 2023-06-27T12:39:31.609 INFO:tasks.rados.rados.0.smithi138.stdout:409: oid 14 version 96 is already newer than 96 2023-06-27T12:39:31.609 INFO:tasks.rados.rados.0.smithi138.stdout:409: oid 14 version 96 is already newer than 96 2023-06-27T12:39:31.609 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 96 (ObjNum 211 snap 45 seq_num 211) dirty exists 2023-06-27T12:39:31.610 INFO:tasks.rados.rados.0.smithi138.stdout:409: left oid 14 (ObjNum 211 snap 45 seq_num 211) 2023-06-27T12:39:31.610 INFO:tasks.rados.rados.0.smithi138.stdout:408: finishing write tid 3 to smithi138138488-13 2023-06-27T12:39:31.610 INFO:tasks.rados.rados.0.smithi138.stdout:408: finishing write tid 4 to smithi138138488-13 2023-06-27T12:39:31.610 INFO:tasks.rados.rados.0.smithi138.stdout:408: finishing write tid 5 to smithi138138488-13 2023-06-27T12:39:31.610 INFO:tasks.rados.rados.0.smithi138.stdout:408: oid 13 updating version 0 to 138 2023-06-27T12:39:31.610 INFO:tasks.rados.rados.0.smithi138.stdout:408: oid 13 version 138 is already newer than 138 2023-06-27T12:39:31.610 INFO:tasks.rados.rados.0.smithi138.stdout:408: oid 13 version 138 is already newer than 138 2023-06-27T12:39:31.611 INFO:tasks.rados.rados.0.smithi138.stdout:408: oid 13 version 138 is already newer than 138 2023-06-27T12:39:31.611 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 138 (ObjNum 210 snap 45 seq_num 210) dirty exists 2023-06-27T12:39:31.611 INFO:tasks.rados.rados.0.smithi138.stdout:408: left oid 13 (ObjNum 210 snap 45 seq_num 210) 2023-06-27T12:39:31.687 INFO:tasks.rados.rados.0.smithi138.stdout:412: finishing write tid 1 to smithi138138488-31 2023-06-27T12:39:31.708 INFO:tasks.rados.rados.0.smithi138.stdout:412: finishing write tid 2 to smithi138138488-31 2023-06-27T12:39:31.709 INFO:tasks.rados.rados.0.smithi138.stdout:412: finishing write tid 3 to smithi138138488-31 2023-06-27T12:39:31.710 INFO:tasks.rados.rados.0.smithi138.stdout:412: finishing write tid 4 to smithi138138488-31 2023-06-27T12:39:31.711 INFO:tasks.rados.rados.0.smithi138.stdout:412: finishing write tid 5 to smithi138138488-31 2023-06-27T12:39:31.711 INFO:tasks.rados.rados.0.smithi138.stdout:412: oid 31 updating version 0 to 99 2023-06-27T12:39:31.711 INFO:tasks.rados.rados.0.smithi138.stdout:412: oid 31 version 99 is already newer than 99 2023-06-27T12:39:31.711 INFO:tasks.rados.rados.0.smithi138.stdout:412: oid 31 version 99 is already newer than 96 2023-06-27T12:39:31.712 INFO:tasks.rados.rados.0.smithi138.stdout:412: oid 31 version 99 is already newer than 99 2023-06-27T12:39:31.712 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 99 (ObjNum 212 snap 45 seq_num 212) dirty exists 2023-06-27T12:39:31.712 INFO:tasks.rados.rados.0.smithi138.stdout:412: left oid 31 (ObjNum 212 snap 45 seq_num 212) 2023-06-27T12:39:32.586 INFO:tasks.rados.rados.0.smithi138.stdout:408: done (4 left) 2023-06-27T12:39:32.586 INFO:tasks.rados.rados.0.smithi138.stdout:409: done (3 left) 2023-06-27T12:39:32.587 INFO:tasks.rados.rados.0.smithi138.stdout:410: done (2 left) 2023-06-27T12:39:32.587 INFO:tasks.rados.rados.0.smithi138.stdout:412: done (1 left) 2023-06-27T12:39:32.587 INFO:tasks.rados.rados.0.smithi138.stdout:413: done (0 left) 2023-06-27T12:39:32.587 INFO:tasks.rados.rados.0.smithi138.stdout:414: snap_remove snap 43 2023-06-27T12:39:33.588 INFO:tasks.rados.rados.0.smithi138.stdout:414: done (0 left) 2023-06-27T12:39:33.588 INFO:tasks.rados.rados.0.smithi138.stdout:415: read oid 22 snap -1 2023-06-27T12:39:33.588 INFO:tasks.rados.rados.0.smithi138.stdout:415: expect (ObjNum 194 snap 40 seq_num 194) 2023-06-27T12:39:33.589 INFO:tasks.rados.rados.0.smithi138.stdout:416: snap_remove snap 38 2023-06-27T12:39:33.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:33.737 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:33.732+0000 7f4613071700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:33.740 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:33.736+0000 7f4613071700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:33.943 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:33.943 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:33.943 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:34.074 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:34.067+0000 7f1dde8dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:34.077 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:34.073+0000 7f1dde8dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:34.268 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:34.270 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:34.282 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2023-06-27T12:39:33.924661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103986867,"num_objects":140,"num_object_clones":86,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":739,"num_read_kb":433361,"num_write":838,"num_write_kb":203715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":940,"ondisk_log_size":940,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":505648,"kb_used_data":389644,"kb_used_omap":0,"kb_used_meta":115456,"kb_avail":376981712,"statfs":{"total":386547056640,"available":386029273088,"internally_reserved":0,"allocated":398995456,"data_stored":440474033,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":118226944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000603"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":183,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.569353+0000","last_change":"2023-06-27T12:39:27.557908+0000","last_active":"2023-06-27T12:39:28.569353+0000","last_peered":"2023-06-27T12:39:28.569353+0000","last_clean":"2023-06-27T12:39:28.569353+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:28.569353+0000","last_undegraded":"2023-06-27T12:39:28.569353+0000","last_fullsized":"2023-06-27T12:39:28.569353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"99'73","reported_seq":279,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.584784+0000","last_change":"2023-06-27T12:39:27.580168+0000","last_active":"2023-06-27T12:39:28.584784+0000","last_peered":"2023-06-27T12:39:28.584784+0000","last_clean":"2023-06-27T12:39:28.584784+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:28.584784+0000","last_undegraded":"2023-06-27T12:39:28.584784+0000","last_fullsized":"2023-06-27T12:39:28.584784+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9176622,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":45,"num_read_kb":37076,"num_write":63,"num_write_kb":18263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"95'67","reported_seq":261,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.823890+0000","last_change":"2023-06-27T12:39:27.595828+0000","last_active":"2023-06-27T12:39:28.823890+0000","last_peered":"2023-06-27T12:39:28.823890+0000","last_clean":"2023-06-27T12:39:28.823890+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:28.823890+0000","last_undegraded":"2023-06-27T12:39:28.823890+0000","last_fullsized":"2023-06-27T12:39:28.823890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9613604,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":20445,"num_write":61,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'40","reported_seq":230,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.574664+0000","last_change":"2023-06-27T12:39:27.659662+0000","last_active":"2023-06-27T12:39:28.574664+0000","last_peered":"2023-06-27T12:39:28.574664+0000","last_clean":"2023-06-27T12:39:28.574664+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:28.574664+0000","last_undegraded":"2023-06-27T12:39:28.574664+0000","last_fullsized":"2023-06-27T12:39:28.574664+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395047,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":38,"num_read_kb":13466,"num_write":35,"num_write_kb":10330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":209,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.596166+0000","last_change":"2023-06-27T12:39:27.852264+0000","last_active":"2023-06-27T12:39:28.596166+0000","last_peered":"2023-06-27T12:39:28.596166+0000","last_clean":"2023-06-27T12:39:28.596166+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:28.596166+0000","last_undegraded":"2023-06-27T12:39:28.596166+0000","last_fullsized":"2023-06-27T12:39:28.596166+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":19,"num_read_kb":14491,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"95'84","reported_seq":293,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.822879+0000","last_change":"2023-06-27T12:39:27.559058+0000","last_active":"2023-06-27T12:39:28.822879+0000","last_peered":"2023-06-27T12:39:28.822879+0000","last_clean":"2023-06-27T12:39:28.822879+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:28.822879+0000","last_undegraded":"2023-06-27T12:39:28.822879+0000","last_fullsized":"2023-06-27T12:39:28.822879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13176655,"num_objects":16,"num_object_clones":10,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":112,"num_read_kb":76828,"num_write":72,"num_write_kb":20414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":254,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.570361+0000","last_change":"2023-06-27T12:39:27.588520+0000","last_active":"2023-06-27T12:39:28.570361+0000","last_peered":"2023-06-27T12:39:28.570361+0000","last_clean":"2023-06-27T12:39:28.570361+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:28.570361+0000","last_undegraded":"2023-06-27T12:39:28.570361+0000","last_fullsized":"2023-06-27T12:39:28.570361+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":24452,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"65'14","reported_seq":209,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.810351+0000","last_change":"2023-06-27T12:39:27.584184+0000","last_active":"2023-06-27T12:39:28.810351+0000","last_peered":"2023-06-27T12:39:28.810351+0000","last_clean":"2023-06-27T12:39:28.810351+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:28.810351+0000","last_undegraded":"2023-06-27T12:39:28.810351+0000","last_fullsized":"2023-06-27T12:39:28.810351+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2705927,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":25158,"num_write":13,"num_write_kb":3961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":155,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.578239+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:28.578239+0000","last_peered":"2023-06-27T12:39:28.578239+0000","last_clean":"2023-06-27T12:39:28.578239+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:28.578239+0000","last_undegraded":"2023-06-27T12:39:28.578239+0000","last_fullsized":"2023-06-27T12:39:28.578239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":182,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.826529+0000","last_change":"2023-06-27T12:39:27.597268+0000","last_active":"2023-06-27T12:39:28.826529+0000","last_peered":"2023-06-27T12:39:28.826529+0000","last_clean":"2023-06-27T12:39:28.826529+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:28.826529+0000","last_undegraded":"2023-06-27T12:39:28.826529+0000","last_fullsized":"2023-06-27T12:39:28.826529+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":145,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.566280+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:28.566280+0000","last_peered":"2023-06-27T12:39:28.566280+0000","last_clean":"2023-06-27T12:39:28.566280+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:28.566280+0000","last_undegraded":"2023-06-27T12:39:28.566280+0000","last_fullsized":"2023-06-27T12:39:28.566280+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"100'51","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.860869+0000","last_change":"2023-06-27T12:39:27.578936+0000","last_active":"2023-06-27T12:39:28.860869+0000","last_peered":"2023-06-27T12:39:28.860869+0000","last_clean":"2023-06-27T12:39:28.860869+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:28.860869+0000","last_undegraded":"2023-06-27T12:39:28.860869+0000","last_fullsized":"2023-06-27T12:39:28.860869+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168169,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":28,"num_read_kb":20337,"num_write":38,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":149,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.820134+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:28.820134+0000","last_peered":"2023-06-27T12:39:28.820134+0000","last_clean":"2023-06-27T12:39:28.820134+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:28.820134+0000","last_undegraded":"2023-06-27T12:39:28.820134+0000","last_fullsized":"2023-06-27T12:39:28.820134+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"88'125","reported_seq":302,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.598756+0000","last_change":"2023-06-27T12:39:27.807485+0000","last_active":"2023-06-27T12:39:28.598756+0000","last_peered":"2023-06-27T12:39:28.598756+0000","last_clean":"2023-06-27T12:39:28.598756+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:28.598756+0000","last_undegraded":"2023-06-27T12:39:28.598756+0000","last_fullsized":"2023-06-27T12:39:28.598756+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8605184,"num_objects":16,"num_object_clones":11,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":68,"num_read_kb":62728,"num_write":95,"num_write_kb":27990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":149,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.830643+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:28.830643+0000","last_peered":"2023-06-27T12:39:28.830643+0000","last_clean":"2023-06-27T12:39:28.830643+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:28.830643+0000","last_undegraded":"2023-06-27T12:39:28.830643+0000","last_fullsized":"2023-06-27T12:39:28.830643+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":270,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.593895+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:28.593895+0000","last_peered":"2023-06-27T12:39:28.593895+0000","last_clean":"2023-06-27T12:39:28.593895+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:28.593895+0000","last_undegraded":"2023-06-27T12:39:28.593895+0000","last_fullsized":"2023-06-27T12:39:28.593895+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"95'91","reported_seq":300,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.832418+0000","last_change":"2023-06-27T12:39:27.585131+0000","last_active":"2023-06-27T12:39:28.832418+0000","last_peered":"2023-06-27T12:39:28.832418+0000","last_clean":"2023-06-27T12:39:28.832418+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:28.832418+0000","last_undegraded":"2023-06-27T12:39:28.832418+0000","last_fullsized":"2023-06-27T12:39:28.832418+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15176655,"num_objects":15,"num_object_clones":9,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":90,"num_read_kb":38297,"num_write":83,"num_write_kb":23556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":145,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.573105+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:28.573105+0000","last_peered":"2023-06-27T12:39:28.573105+0000","last_clean":"2023-06-27T12:39:28.573105+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:28.573105+0000","last_undegraded":"2023-06-27T12:39:28.573105+0000","last_fullsized":"2023-06-27T12:39:28.573105+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"100'50","reported_seq":227,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.801355+0000","last_change":"2023-06-27T12:39:27.804969+0000","last_active":"2023-06-27T12:39:28.801355+0000","last_peered":"2023-06-27T12:39:28.801355+0000","last_clean":"2023-06-27T12:39:28.801355+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:28.801355+0000","last_undegraded":"2023-06-27T12:39:28.801355+0000","last_fullsized":"2023-06-27T12:39:28.801355+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3428628,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":27,"num_read_kb":24672,"num_write":40,"num_write_kb":11607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.586416+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:28.586416+0000","last_peered":"2023-06-27T12:39:28.586416+0000","last_clean":"2023-06-27T12:39:28.586416+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:28.586416+0000","last_undegraded":"2023-06-27T12:39:28.586416+0000","last_fullsized":"2023-06-27T12:39:28.586416+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"80'50","reported_seq":254,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.603384+0000","last_change":"2023-06-27T12:39:27.574819+0000","last_active":"2023-06-27T12:39:28.603384+0000","last_peered":"2023-06-27T12:39:28.603384+0000","last_clean":"2023-06-27T12:39:28.603384+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:28.603384+0000","last_undegraded":"2023-06-27T12:39:28.603384+0000","last_fullsized":"2023-06-27T12:39:28.603384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10453888,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":34,"num_read_kb":35970,"num_write":46,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"82'35","reported_seq":212,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.588992+0000","last_change":"2023-06-27T12:39:27.804633+0000","last_active":"2023-06-27T12:39:28.588992+0000","last_peered":"2023-06-27T12:39:28.588992+0000","last_clean":"2023-06-27T12:39:28.588992+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:28.588992+0000","last_undegraded":"2023-06-27T12:39:28.588992+0000","last_fullsized":"2023-06-27T12:39:28.588992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285752,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":17,"num_read_kb":13728,"num_write":30,"num_write_kb":8133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":155,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.586946+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:28.586946+0000","last_peered":"2023-06-27T12:39:28.586946+0000","last_clean":"2023-06-27T12:39:28.586946+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:28.586946+0000","last_undegraded":"2023-06-27T12:39:28.586946+0000","last_fullsized":"2023-06-27T12:39:28.586946+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.811760+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:28.811760+0000","last_peered":"2023-06-27T12:39:28.811760+0000","last_clean":"2023-06-27T12:39:28.811760+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:28.811760+0000","last_undegraded":"2023-06-27T12:39:28.811760+0000","last_fullsized":"2023-06-27T12:39:28.811760+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"95'55","reported_seq":246,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-06-27T12:39:28.571661+0000","last_change":"2023-06-27T12:39:27.575105+0000","last_active":"2023-06-27T12:39:28.571661+0000","last_peered":"2023-06-27T12:39:28.571661+0000","last_clean":"2023-06-27T12:39:28.571661+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:28.571661+0000","last_undegraded":"2023-06-27T12:39:28.571661+0000","last_fullsized":"2023-06-27T12:39:28.571661+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8067360,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":25621,"num_write":40,"num_write_kb":11503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":103396499,"num_objects":138,"num_object_clones":86,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":636,"num_read_kb":433270,"num_write":701,"num_write_kb":201821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":843,"ondisk_log_size":843,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673055,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":134488,"kb_used_data":103568,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94237352,"statfs":{"total":96636764160,"available":96499048448,"internally_reserved":0,"allocated":106053632,"data_stored":116436570,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673055,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":122200,"kb_used_data":91280,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94249640,"statfs":{"total":96636764160,"available":96511631360,"internally_reserved":0,"allocated":93470720,"data_stored":102093892,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705760,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":118440,"kb_used_data":91616,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94253400,"statfs":{"total":96636764160,"available":96515481600,"internally_reserved":0,"allocated":93814784,"data_stored":103092438,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673055,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":130520,"kb_used_data":103180,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94241320,"statfs":{"total":96636764160,"available":96503111680,"internally_reserved":0,"allocated":105656320,"data_stored":118851133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:34.595 INFO:tasks.rados.rados.0.smithi138.stdout:416: done (1 left) 2023-06-27T12:39:34.595 INFO:tasks.rados.rados.0.smithi138.stdout:417: write (excl) oid 14 current snap is 46 2023-06-27T12:39:34.595 INFO:tasks.rados.rados.0.smithi138.stdout:417: seq_num 213 ranges {779891=485921,1947841=685395,3383215=196676} 2023-06-27T12:39:34.602 INFO:tasks.rados.rados.0.smithi138.stdout:417: writing smithi138138488-14 from 779891 to 1265812 tid 1 2023-06-27T12:39:34.613 INFO:tasks.rados.rados.0.smithi138.stdout:417: writing smithi138138488-14 from 1947841 to 2633236 tid 2 2023-06-27T12:39:34.619 INFO:tasks.rados.rados.0.smithi138.stdout:417: writing smithi138138488-14 from 3383215 to 3579891 tid 3 2023-06-27T12:39:34.619 INFO:tasks.rados.rados.0.smithi138.stdout:418: write oid 11 current snap is 46 2023-06-27T12:39:34.619 INFO:tasks.rados.rados.0.smithi138.stdout:418: seq_num 214 ranges {796698=561170,2006323=545406,3126991=469707} 2023-06-27T12:39:34.629 INFO:tasks.rados.rados.0.smithi138.stdout:418: writing smithi138138488-11 from 796698 to 1357868 tid 1 2023-06-27T12:39:34.638 INFO:tasks.rados.rados.0.smithi138.stdout:418: writing smithi138138488-11 from 2006323 to 2551729 tid 2 2023-06-27T12:39:34.646 INFO:tasks.rados.rados.0.smithi138.stdout:418: writing smithi138138488-11 from 3126991 to 3596698 tid 3 2023-06-27T12:39:34.647 INFO:tasks.rados.rados.0.smithi138.stdout:419: read oid 30 snap -1 2023-06-27T12:39:34.647 INFO:tasks.rados.rados.0.smithi138.stdout:419: expect deleted 2023-06-27T12:39:34.647 INFO:tasks.rados.rados.0.smithi138.stdout:420: snap_create 2023-06-27T12:39:34.647 INFO:tasks.rados.rados.0.smithi138.stdout:415: expect (ObjNum 194 snap 40 seq_num 194) 2023-06-27T12:39:34.780 INFO:tasks.rados.rados.0.smithi138.stdout:417: finishing write tid 1 to smithi138138488-14 2023-06-27T12:39:34.780 INFO:tasks.rados.rados.0.smithi138.stdout:418: finishing write tid 1 to smithi138138488-11 2023-06-27T12:39:34.781 INFO:tasks.rados.rados.0.smithi138.stdout:418: finishing write tid 2 to smithi138138488-11 2023-06-27T12:39:34.781 INFO:tasks.rados.rados.0.smithi138.stdout:417: finishing write tid 2 to smithi138138488-14 2023-06-27T12:39:34.781 INFO:tasks.rados.rados.0.smithi138.stdout:417: finishing write tid 3 to smithi138138488-14 2023-06-27T12:39:34.781 INFO:tasks.rados.rados.0.smithi138.stdout:417: finishing write tid 4 to smithi138138488-14 2023-06-27T12:39:34.781 INFO:tasks.rados.rados.0.smithi138.stdout:417: finishing write tid 5 to smithi138138488-14 2023-06-27T12:39:34.781 INFO:tasks.rados.rados.0.smithi138.stdout:417: oid 14 updating version 0 to 101 2023-06-27T12:39:34.781 INFO:tasks.rados.rados.0.smithi138.stdout:417: oid 14 updating version 101 to 105 2023-06-27T12:39:34.781 INFO:tasks.rados.rados.0.smithi138.stdout:417: oid 14 version 105 is already newer than 105 2023-06-27T12:39:34.782 INFO:tasks.rados.rados.0.smithi138.stdout:417: oid 14 version 105 is already newer than 105 2023-06-27T12:39:34.782 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 105 (ObjNum 213 snap 46 seq_num 213) dirty exists 2023-06-27T12:39:34.782 INFO:tasks.rados.rados.0.smithi138.stdout:417: left oid 14 (ObjNum 213 snap 46 seq_num 213) 2023-06-27T12:39:34.782 INFO:tasks.rados.rados.0.smithi138.stdout:418: finishing write tid 3 to smithi138138488-11 2023-06-27T12:39:34.782 INFO:tasks.rados.rados.0.smithi138.stdout:418: finishing write tid 4 to smithi138138488-11 2023-06-27T12:39:34.782 INFO:tasks.rados.rados.0.smithi138.stdout:418: finishing write tid 5 to smithi138138488-11 2023-06-27T12:39:34.782 INFO:tasks.rados.rados.0.smithi138.stdout:418: oid 11 updating version 0 to 71 2023-06-27T12:39:34.783 INFO:tasks.rados.rados.0.smithi138.stdout:418: oid 11 version 71 is already newer than 70 2023-06-27T12:39:34.783 INFO:tasks.rados.rados.0.smithi138.stdout:418: oid 11 version 71 is already newer than 71 2023-06-27T12:39:34.783 INFO:tasks.rados.rados.0.smithi138.stdout:418: oid 11 version 71 is already newer than 71 2023-06-27T12:39:34.783 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 11 v 71 (ObjNum 214 snap 46 seq_num 214) dirty exists 2023-06-27T12:39:34.783 INFO:tasks.rados.rados.0.smithi138.stdout:418: left oid 11 (ObjNum 214 snap 46 seq_num 214) 2023-06-27T12:39:35.599 INFO:tasks.rados.rados.0.smithi138.stdout:415: done (4 left) 2023-06-27T12:39:35.600 INFO:tasks.rados.rados.0.smithi138.stdout:417: done (3 left) 2023-06-27T12:39:35.600 INFO:tasks.rados.rados.0.smithi138.stdout:418: done (2 left) 2023-06-27T12:39:35.600 INFO:tasks.rados.rados.0.smithi138.stdout:419: done (1 left) 2023-06-27T12:39:35.600 INFO:tasks.rados.rados.0.smithi138.stdout:420: done (0 left) 2023-06-27T12:39:35.600 INFO:tasks.rados.rados.0.smithi138.stdout:421: write (excl) oid 13 current snap is 47 2023-06-27T12:39:35.600 INFO:tasks.rados.rados.0.smithi138.stdout:421: seq_num 215 ranges {413505=636419,1664805=689064,3038061=492071} 2023-06-27T12:39:35.605 INFO:tasks.rados.rados.0.smithi138.stdout:421: writing smithi138138488-13 from 413505 to 1049924 tid 1 2023-06-27T12:39:35.612 INFO:tasks.rados.rados.0.smithi138.stdout:421: writing smithi138138488-13 from 1664805 to 2353869 tid 2 2023-06-27T12:39:35.618 INFO:tasks.rados.rados.0.smithi138.stdout:421: writing smithi138138488-13 from 3038061 to 3530132 tid 3 2023-06-27T12:39:35.619 INFO:tasks.rados.rados.0.smithi138.stdout:422: write oid 48 current snap is 47 2023-06-27T12:39:35.619 INFO:tasks.rados.rados.0.smithi138.stdout:422: seq_num 216 ranges {430312=711668,1723287=549075,2781837=703343} 2023-06-27T12:39:35.625 INFO:tasks.rados.rados.0.smithi138.stdout:422: writing smithi138138488-48 from 430312 to 1141980 tid 1 2023-06-27T12:39:35.631 INFO:tasks.rados.rados.0.smithi138.stdout:422: writing smithi138138488-48 from 1723287 to 2272362 tid 2 2023-06-27T12:39:35.638 INFO:tasks.rados.rados.0.smithi138.stdout:422: writing smithi138138488-48 from 2781837 to 3485180 tid 3 2023-06-27T12:39:35.638 INFO:tasks.rados.rados.0.smithi138.stdout:423: write (excl) oid 12 current snap is 47 2023-06-27T12:39:35.638 INFO:tasks.rados.rados.0.smithi138.stdout:423: seq_num 217 ranges {447119=786917,1665416=692733,2976554=514615} 2023-06-27T12:39:35.644 INFO:tasks.rados.rados.0.smithi138.stdout:423: writing smithi138138488-12 from 447119 to 1234036 tid 1 2023-06-27T12:39:35.653 INFO:tasks.rados.rados.0.smithi138.stdout:423: writing smithi138138488-12 from 1665416 to 2358149 tid 2 2023-06-27T12:39:35.662 INFO:tasks.rados.rados.0.smithi138.stdout:423: writing smithi138138488-12 from 2976554 to 3491169 tid 3 2023-06-27T12:39:35.662 INFO:tasks.rados.rados.0.smithi138.stdout:424: snap_remove snap 45 2023-06-27T12:39:35.816 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:35.941 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:35.936+0000 7f1a60170700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:35.944 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:35.941+0000 7f1a60170700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:36.604 INFO:tasks.rados.rados.0.smithi138.stdout:424: done (3 left) 2023-06-27T12:39:36.604 INFO:tasks.rados.rados.0.smithi138.stdout:425: snap_create 2023-06-27T12:39:36.605 INFO:tasks.rados.rados.0.smithi138.stdout:422: finishing write tid 1 to smithi138138488-48 2023-06-27T12:39:36.605 INFO:tasks.rados.rados.0.smithi138.stdout:422: finishing write tid 2 to smithi138138488-48 2023-06-27T12:39:36.605 INFO:tasks.rados.rados.0.smithi138.stdout:421: finishing write tid 1 to smithi138138488-13 2023-06-27T12:39:36.605 INFO:tasks.rados.rados.0.smithi138.stdout:422: finishing write tid 3 to smithi138138488-48 2023-06-27T12:39:36.605 INFO:tasks.rados.rados.0.smithi138.stdout:422: finishing write tid 4 to smithi138138488-48 2023-06-27T12:39:36.605 INFO:tasks.rados.rados.0.smithi138.stdout:423: finishing write tid 1 to smithi138138488-12 2023-06-27T12:39:36.605 INFO:tasks.rados.rados.0.smithi138.stdout:422: finishing write tid 5 to smithi138138488-48 2023-06-27T12:39:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:422: oid 48 updating version 0 to 60 2023-06-27T12:39:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:422: oid 48 version 60 is already newer than 60 2023-06-27T12:39:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:422: oid 48 version 60 is already newer than 60 2023-06-27T12:39:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:422: oid 48 version 60 is already newer than 60 2023-06-27T12:39:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 60 (ObjNum 216 snap 47 seq_num 216) dirty exists 2023-06-27T12:39:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:422: left oid 48 (ObjNum 216 snap 47 seq_num 216) 2023-06-27T12:39:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:423: finishing write tid 2 to smithi138138488-12 2023-06-27T12:39:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:421: finishing write tid 2 to smithi138138488-13 2023-06-27T12:39:36.607 INFO:tasks.rados.rados.0.smithi138.stdout:421: finishing write tid 3 to smithi138138488-13 2023-06-27T12:39:36.607 INFO:tasks.rados.rados.0.smithi138.stdout:421: finishing write tid 4 to smithi138138488-13 2023-06-27T12:39:36.607 INFO:tasks.rados.rados.0.smithi138.stdout:421: finishing write tid 5 to smithi138138488-13 2023-06-27T12:39:36.607 INFO:tasks.rados.rados.0.smithi138.stdout:421: oid 13 updating version 0 to 143 2023-06-27T12:39:36.607 INFO:tasks.rados.rados.0.smithi138.stdout:421: oid 13 version 143 is already newer than 143 2023-06-27T12:39:36.607 INFO:tasks.rados.rados.0.smithi138.stdout:421: oid 13 version 143 is already newer than 143 2023-06-27T12:39:36.607 INFO:tasks.rados.rados.0.smithi138.stdout:421: oid 13 version 143 is already newer than 139 2023-06-27T12:39:36.607 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 143 (ObjNum 215 snap 47 seq_num 215) dirty exists 2023-06-27T12:39:36.608 INFO:tasks.rados.rados.0.smithi138.stdout:421: left oid 13 (ObjNum 215 snap 47 seq_num 215) 2023-06-27T12:39:36.608 INFO:tasks.rados.rados.0.smithi138.stdout:423: finishing write tid 3 to smithi138138488-12 2023-06-27T12:39:36.608 INFO:tasks.rados.rados.0.smithi138.stdout:423: finishing write tid 4 to smithi138138488-12 2023-06-27T12:39:36.608 INFO:tasks.rados.rados.0.smithi138.stdout:423: finishing write tid 5 to smithi138138488-12 2023-06-27T12:39:36.608 INFO:tasks.rados.rados.0.smithi138.stdout:423: oid 12 updating version 0 to 72 2023-06-27T12:39:36.608 INFO:tasks.rados.rados.0.smithi138.stdout:423: oid 12 updating version 72 to 76 2023-06-27T12:39:36.608 INFO:tasks.rados.rados.0.smithi138.stdout:423: oid 12 version 76 is already newer than 76 2023-06-27T12:39:36.609 INFO:tasks.rados.rados.0.smithi138.stdout:423: oid 12 version 76 is already newer than 76 2023-06-27T12:39:36.609 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 76 (ObjNum 217 snap 47 seq_num 217) dirty exists 2023-06-27T12:39:36.609 INFO:tasks.rados.rados.0.smithi138.stdout:423: left oid 12 (ObjNum 217 snap 47 seq_num 217) 2023-06-27T12:39:37.605 INFO:tasks.rados.rados.0.smithi138.stdout:421: done (3 left) 2023-06-27T12:39:37.605 INFO:tasks.rados.rados.0.smithi138.stdout:422: done (2 left) 2023-06-27T12:39:37.606 INFO:tasks.rados.rados.0.smithi138.stdout:423: done (1 left) 2023-06-27T12:39:37.606 INFO:tasks.rados.rados.0.smithi138.stdout:425: done (0 left) 2023-06-27T12:39:37.606 INFO:tasks.rados.rados.0.smithi138.stdout:426: snap_create 2023-06-27T12:39:38.015 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:38.153 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:38.149+0000 7f56fe8eb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:38.157 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:38.153+0000 7f56fe8eb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:38.606 INFO:tasks.rados.rados.0.smithi138.stdout:426: done (0 left) 2023-06-27T12:39:38.606 INFO:tasks.rados.rados.0.smithi138.stdout:427: write (excl) oid 14 current snap is 49 2023-06-27T12:39:38.606 INFO:tasks.rados.rados.0.smithi138.stdout:427: seq_num 218 ranges {463926=462166,1723898=436391,2603977=725887} 2023-06-27T12:39:38.610 INFO:tasks.rados.rados.0.smithi138.stdout:427: writing smithi138138488-14 from 463926 to 926092 tid 1 2023-06-27T12:39:38.620 INFO:tasks.rados.rados.0.smithi138.stdout:427: writing smithi138138488-14 from 1723898 to 2160289 tid 2 2023-06-27T12:39:38.630 INFO:tasks.rados.rados.0.smithi138.stdout:427: writing smithi138138488-14 from 2603977 to 3329864 tid 3 2023-06-27T12:39:38.630 INFO:tasks.rados.rados.0.smithi138.stdout:428: write (excl) oid 17 current snap is 49 2023-06-27T12:39:38.631 INFO:tasks.rados.rados.0.smithi138.stdout:428: seq_num 219 ranges {480733=537415,1782380=696402,3031400=537159} 2023-06-27T12:39:38.637 INFO:tasks.rados.rados.0.smithi138.stdout:428: writing smithi138138488-17 from 480733 to 1018148 tid 1 2023-06-27T12:39:38.648 INFO:tasks.rados.rados.0.smithi138.stdout:428: writing smithi138138488-17 from 1782380 to 2478782 tid 2 2023-06-27T12:39:38.657 INFO:tasks.rados.rados.0.smithi138.stdout:428: writing smithi138138488-17 from 3031400 to 3568559 tid 3 2023-06-27T12:39:38.657 INFO:tasks.rados.rados.0.smithi138.stdout:429: write oid 24 current snap is 49 2023-06-27T12:39:38.658 INFO:tasks.rados.rados.0.smithi138.stdout:429: seq_num 220 ranges {497540=612664,1840862=440060,3058823=464784} 2023-06-27T12:39:38.667 INFO:tasks.rados.rados.0.smithi138.stdout:429: writing smithi138138488-24 from 497540 to 1110204 tid 1 2023-06-27T12:39:38.675 INFO:tasks.rados.rados.0.smithi138.stdout:429: writing smithi138138488-24 from 1840862 to 2280922 tid 2 2023-06-27T12:39:38.684 INFO:tasks.rados.rados.0.smithi138.stdout:429: writing smithi138138488-24 from 3058823 to 3523607 tid 3 2023-06-27T12:39:38.685 INFO:tasks.rados.rados.0.smithi138.stdout:430: snap_remove snap 41 2023-06-27T12:39:39.285 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:39.285 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:39.285 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:39.416 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:39.409+0000 7f7c243b7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:39.415+0000 7f7c243b7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:39.612 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:39.620 INFO:tasks.rados.rados.0.smithi138.stdout:430: done (3 left) 2023-06-27T12:39:39.621 INFO:tasks.rados.rados.0.smithi138.stdout:431: snap_create 2023-06-27T12:39:39.621 INFO:tasks.rados.rados.0.smithi138.stdout:428: finishing write tid 1 to smithi138138488-17 2023-06-27T12:39:39.621 INFO:tasks.rados.rados.0.smithi138.stdout:428: finishing write tid 2 to smithi138138488-17 2023-06-27T12:39:39.621 INFO:tasks.rados.rados.0.smithi138.stdout:427: finishing write tid 1 to smithi138138488-14 2023-06-27T12:39:39.621 INFO:tasks.rados.rados.0.smithi138.stdout:428: finishing write tid 3 to smithi138138488-17 2023-06-27T12:39:39.621 INFO:tasks.rados.rados.0.smithi138.stdout:428: finishing write tid 4 to smithi138138488-17 2023-06-27T12:39:39.622 INFO:tasks.rados.rados.0.smithi138.stdout:428: finishing write tid 5 to smithi138138488-17 2023-06-27T12:39:39.622 INFO:tasks.rados.rados.0.smithi138.stdout:428: oid 17 updating version 0 to 80 2023-06-27T12:39:39.622 INFO:tasks.rados.rados.0.smithi138.stdout:428: oid 17 updating version 80 to 84 2023-06-27T12:39:39.622 INFO:tasks.rados.rados.0.smithi138.stdout:428: oid 17 version 84 is already newer than 84 2023-06-27T12:39:39.622 INFO:tasks.rados.rados.0.smithi138.stdout:428: oid 17 version 84 is already newer than 84 2023-06-27T12:39:39.622 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 84 (ObjNum 219 snap 49 seq_num 219) dirty exists 2023-06-27T12:39:39.622 INFO:tasks.rados.rados.0.smithi138.stdout:428: left oid 17 (ObjNum 219 snap 49 seq_num 219) 2023-06-27T12:39:39.622 INFO:tasks.rados.rados.0.smithi138.stdout:429: finishing write tid 1 to smithi138138488-24 2023-06-27T12:39:39.623 INFO:tasks.rados.rados.0.smithi138.stdout:429: finishing write tid 2 to smithi138138488-24 2023-06-27T12:39:39.623 INFO:tasks.rados.rados.0.smithi138.stdout:429: finishing write tid 3 to smithi138138488-24 2023-06-27T12:39:39.623 INFO:tasks.rados.rados.0.smithi138.stdout:427: finishing write tid 2 to smithi138138488-14 2023-06-27T12:39:39.623 INFO:tasks.rados.rados.0.smithi138.stdout:429: finishing write tid 4 to smithi138138488-24 2023-06-27T12:39:39.623 INFO:tasks.rados.rados.0.smithi138.stdout:429: finishing write tid 5 to smithi138138488-24 2023-06-27T12:39:39.623 INFO:tasks.rados.rados.0.smithi138.stdout:429: oid 24 updating version 0 to 112 2023-06-27T12:39:39.623 INFO:tasks.rados.rados.0.smithi138.stdout:429: oid 24 version 112 is already newer than 112 2023-06-27T12:39:39.624 INFO:tasks.rados.rados.0.smithi138.stdout:429: oid 24 version 112 is already newer than 112 2023-06-27T12:39:39.624 INFO:tasks.rados.rados.0.smithi138.stdout:429: oid 24 version 112 is already newer than 112 2023-06-27T12:39:39.624 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 112 (ObjNum 220 snap 49 seq_num 220) dirty exists 2023-06-27T12:39:39.624 INFO:tasks.rados.rados.0.smithi138.stdout:429: left oid 24 (ObjNum 220 snap 49 seq_num 220) 2023-06-27T12:39:39.624 INFO:tasks.rados.rados.0.smithi138.stdout:427: finishing write tid 3 to smithi138138488-14 2023-06-27T12:39:39.624 INFO:tasks.rados.rados.0.smithi138.stdout:427: finishing write tid 4 to smithi138138488-14 2023-06-27T12:39:39.624 INFO:tasks.rados.rados.0.smithi138.stdout:427: finishing write tid 5 to smithi138138488-14 2023-06-27T12:39:39.625 INFO:tasks.rados.rados.0.smithi138.stdout:427: oid 14 updating version 0 to 108 2023-06-27T12:39:39.625 INFO:tasks.rados.rados.0.smithi138.stdout:427: oid 14 updating version 108 to 112 2023-06-27T12:39:39.625 INFO:tasks.rados.rados.0.smithi138.stdout:427: oid 14 version 112 is already newer than 112 2023-06-27T12:39:39.625 INFO:tasks.rados.rados.0.smithi138.stdout:427: oid 14 version 112 is already newer than 112 2023-06-27T12:39:39.625 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 112 (ObjNum 218 snap 49 seq_num 218) dirty exists 2023-06-27T12:39:39.625 INFO:tasks.rados.rados.0.smithi138.stdout:427: left oid 14 (ObjNum 218 snap 49 seq_num 218) 2023-06-27T12:39:39.626 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2023-06-27T12:39:38.601241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105970093,"num_objects":144,"num_object_clones":90,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":752,"num_read_kb":449197,"num_write":863,"num_write_kb":211711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":989,"ondisk_log_size":989,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":533752,"kb_used_data":414168,"kb_used_omap":0,"kb_used_meta":119552,"kb_avail":376953608,"statfs":{"total":386547056640,"available":386000494592,"internally_reserved":0,"allocated":424108032,"data_stored":470954041,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":122421248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.011222"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":196,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.801570+0000","last_change":"2023-06-27T12:39:33.807386+0000","last_active":"2023-06-27T12:39:33.801570+0000","last_peered":"2023-06-27T12:39:33.801570+0000","last_clean":"2023-06-27T12:39:33.801570+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:33.801570+0000","last_undegraded":"2023-06-27T12:39:33.801570+0000","last_fullsized":"2023-06-27T12:39:33.801570+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"99'73","reported_seq":294,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.598157+0000","last_change":"2023-06-27T12:39:33.605161+0000","last_active":"2023-06-27T12:39:33.598157+0000","last_peered":"2023-06-27T12:39:33.598157+0000","last_clean":"2023-06-27T12:39:33.598157+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:33.598157+0000","last_undegraded":"2023-06-27T12:39:33.598157+0000","last_fullsized":"2023-06-27T12:39:33.598157+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9176622,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":43676,"num_write":63,"num_write_kb":18263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"95'67","reported_seq":274,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.605103+0000","last_change":"2023-06-27T12:39:33.609808+0000","last_active":"2023-06-27T12:39:33.605103+0000","last_peered":"2023-06-27T12:39:33.605103+0000","last_clean":"2023-06-27T12:39:33.605103+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:33.605103+0000","last_undegraded":"2023-06-27T12:39:33.605103+0000","last_fullsized":"2023-06-27T12:39:33.605103+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9613604,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":20445,"num_write":61,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'40","reported_seq":244,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.804630+0000","last_change":"2023-06-27T12:39:33.810669+0000","last_active":"2023-06-27T12:39:33.804630+0000","last_peered":"2023-06-27T12:39:33.804630+0000","last_clean":"2023-06-27T12:39:33.804630+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:33.804630+0000","last_undegraded":"2023-06-27T12:39:33.804630+0000","last_fullsized":"2023-06-27T12:39:33.804630+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395047,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":38,"num_read_kb":13466,"num_write":35,"num_write_kb":10330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":224,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.910077+0000","last_change":"2023-06-27T12:39:33.926656+0000","last_active":"2023-06-27T12:39:33.910077+0000","last_peered":"2023-06-27T12:39:33.910077+0000","last_clean":"2023-06-27T12:39:33.910077+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:33.910077+0000","last_undegraded":"2023-06-27T12:39:33.910077+0000","last_fullsized":"2023-06-27T12:39:33.910077+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":21,"num_read_kb":19187,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"103'99","reported_seq":316,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.801795+0000","last_change":"2023-06-27T12:39:33.822786+0000","last_active":"2023-06-27T12:39:33.801795+0000","last_peered":"2023-06-27T12:39:33.801795+0000","last_clean":"2023-06-27T12:39:33.801795+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:33.801795+0000","last_undegraded":"2023-06-27T12:39:33.801795+0000","last_fullsized":"2023-06-27T12:39:33.801795+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17042265,"num_objects":17,"num_object_clones":11,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":115,"num_read_kb":76830,"num_write":82,"num_write_kb":23581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":267,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.798839+0000","last_change":"2023-06-27T12:39:33.810330+0000","last_active":"2023-06-27T12:39:33.798839+0000","last_peered":"2023-06-27T12:39:33.798839+0000","last_clean":"2023-06-27T12:39:33.798839+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:33.798839+0000","last_undegraded":"2023-06-27T12:39:33.798839+0000","last_fullsized":"2023-06-27T12:39:33.798839+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":24452,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"100'23","reported_seq":227,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.598789+0000","last_change":"2023-06-27T12:39:33.660955+0000","last_active":"2023-06-27T12:39:33.598789+0000","last_peered":"2023-06-27T12:39:33.598789+0000","last_clean":"2023-06-27T12:39:33.598789+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:33.598789+0000","last_undegraded":"2023-06-27T12:39:33.598789+0000","last_fullsized":"2023-06-27T12:39:33.598789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3479049,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":25159,"num_write":18,"num_write_kb":5463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.597545+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:33.597545+0000","last_peered":"2023-06-27T12:39:33.597545+0000","last_clean":"2023-06-27T12:39:33.597545+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:33.597545+0000","last_undegraded":"2023-06-27T12:39:33.597545+0000","last_fullsized":"2023-06-27T12:39:33.597545+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":195,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.601513+0000","last_change":"2023-06-27T12:39:33.609305+0000","last_active":"2023-06-27T12:39:33.601513+0000","last_peered":"2023-06-27T12:39:33.601513+0000","last_clean":"2023-06-27T12:39:33.601513+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:33.601513+0000","last_undegraded":"2023-06-27T12:39:33.601513+0000","last_fullsized":"2023-06-27T12:39:33.601513+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.803502+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:33.803502+0000","last_peered":"2023-06-27T12:39:33.803502+0000","last_clean":"2023-06-27T12:39:33.803502+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:33.803502+0000","last_undegraded":"2023-06-27T12:39:33.803502+0000","last_fullsized":"2023-06-27T12:39:33.803502+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"100'51","reported_seq":255,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.600310+0000","last_change":"2023-06-27T12:39:33.606091+0000","last_active":"2023-06-27T12:39:33.600310+0000","last_peered":"2023-06-27T12:39:33.600310+0000","last_clean":"2023-06-27T12:39:33.600310+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:33.600310+0000","last_undegraded":"2023-06-27T12:39:33.600310+0000","last_fullsized":"2023-06-27T12:39:33.600310+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168169,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":28,"num_read_kb":20337,"num_write":38,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":159,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.602370+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:33.602370+0000","last_peered":"2023-06-27T12:39:33.602370+0000","last_clean":"2023-06-27T12:39:33.602370+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:33.602370+0000","last_undegraded":"2023-06-27T12:39:33.602370+0000","last_fullsized":"2023-06-27T12:39:33.602370+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"102'138","reported_seq":322,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.922559+0000","last_change":"2023-06-27T12:39:33.922730+0000","last_active":"2023-06-27T12:39:33.922559+0000","last_peered":"2023-06-27T12:39:33.922559+0000","last_clean":"2023-06-27T12:39:33.922559+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:33.922559+0000","last_undegraded":"2023-06-27T12:39:33.922559+0000","last_fullsized":"2023-06-27T12:39:33.922559+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8974938,"num_objects":17,"num_object_clones":12,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":69,"num_read_kb":62729,"num_write":100,"num_write_kb":29442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.601042+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:33.601042+0000","last_peered":"2023-06-27T12:39:33.601042+0000","last_clean":"2023-06-27T12:39:33.601042+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:33.601042+0000","last_undegraded":"2023-06-27T12:39:33.601042+0000","last_fullsized":"2023-06-27T12:39:33.601042+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":280,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.918907+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:33.918907+0000","last_peered":"2023-06-27T12:39:33.918907+0000","last_clean":"2023-06-27T12:39:33.918907+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:33.918907+0000","last_undegraded":"2023-06-27T12:39:33.918907+0000","last_fullsized":"2023-06-27T12:39:33.918907+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"102'96","reported_seq":320,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.609120+0000","last_change":"2023-06-27T12:39:33.609456+0000","last_active":"2023-06-27T12:39:33.609120+0000","last_peered":"2023-06-27T12:39:33.609120+0000","last_clean":"2023-06-27T12:39:33.609120+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:33.609120+0000","last_undegraded":"2023-06-27T12:39:33.609120+0000","last_fullsized":"2023-06-27T12:39:33.609120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15580023,"num_objects":16,"num_object_clones":10,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":92,"num_read_kb":38298,"num_write":88,"num_write_kb":25431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":155,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.805563+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:33.805563+0000","last_peered":"2023-06-27T12:39:33.805563+0000","last_clean":"2023-06-27T12:39:33.805563+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:33.805563+0000","last_undegraded":"2023-06-27T12:39:33.805563+0000","last_fullsized":"2023-06-27T12:39:33.805563+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"102'51","reported_seq":241,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.919977+0000","last_change":"2023-06-27T12:39:33.981298+0000","last_active":"2023-06-27T12:39:33.919977+0000","last_peered":"2023-06-27T12:39:33.919977+0000","last_clean":"2023-06-27T12:39:33.919977+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:33.919977+0000","last_undegraded":"2023-06-27T12:39:33.919977+0000","last_fullsized":"2023-06-27T12:39:33.919977+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":27,"num_read_kb":24672,"num_write":40,"num_write_kb":11607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":149,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.905322+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:33.905322+0000","last_peered":"2023-06-27T12:39:33.905322+0000","last_clean":"2023-06-27T12:39:33.905322+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:33.905322+0000","last_undegraded":"2023-06-27T12:39:33.905322+0000","last_fullsized":"2023-06-27T12:39:33.905322+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"80'50","reported_seq":267,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.592815+0000","last_change":"2023-06-27T12:39:33.602748+0000","last_active":"2023-06-27T12:39:33.592815+0000","last_peered":"2023-06-27T12:39:33.592815+0000","last_clean":"2023-06-27T12:39:33.592815+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:33.592815+0000","last_undegraded":"2023-06-27T12:39:33.592815+0000","last_fullsized":"2023-06-27T12:39:33.592815+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10453888,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":34,"num_read_kb":35970,"num_write":46,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"100'41","reported_seq":225,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.914801+0000","last_change":"2023-06-27T12:39:33.984820+0000","last_active":"2023-06-27T12:39:33.914801+0000","last_peered":"2023-06-27T12:39:33.914801+0000","last_clean":"2023-06-27T12:39:33.914801+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:33.914801+0000","last_undegraded":"2023-06-27T12:39:33.914801+0000","last_fullsized":"2023-06-27T12:39:33.914801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285752,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":17,"num_read_kb":13728,"num_write":30,"num_write_kb":8133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.593426+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:33.593426+0000","last_peered":"2023-06-27T12:39:33.593426+0000","last_clean":"2023-06-27T12:39:33.593426+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:33.593426+0000","last_undegraded":"2023-06-27T12:39:33.593426+0000","last_fullsized":"2023-06-27T12:39:33.593426+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.602024+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:33.602024+0000","last_peered":"2023-06-27T12:39:33.602024+0000","last_clean":"2023-06-27T12:39:33.602024+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:33.602024+0000","last_undegraded":"2023-06-27T12:39:33.602024+0000","last_fullsized":"2023-06-27T12:39:33.602024+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"95'55","reported_seq":261,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-06-27T12:39:33.607221+0000","last_change":"2023-06-27T12:39:33.601531+0000","last_active":"2023-06-27T12:39:33.607221+0000","last_peered":"2023-06-27T12:39:33.607221+0000","last_clean":"2023-06-27T12:39:33.607221+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:33.607221+0000","last_undegraded":"2023-06-27T12:39:33.607221+0000","last_fullsized":"2023-06-27T12:39:33.607221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8067360,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":30156,"num_write":40,"num_write_kb":11503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":105379725,"num_objects":142,"num_object_clones":90,"num_object_copies":426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":649,"num_read_kb":449106,"num_write":726,"num_write_kb":209817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":892,"ondisk_log_size":892,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":138860,"kb_used_data":107940,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94232980,"statfs":{"total":96636764160,"available":96494571520,"internally_reserved":0,"allocated":110530560,"data_stored":122171458,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":129324,"kb_used_data":98404,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94242516,"statfs":{"total":96636764160,"available":96504336384,"internally_reserved":0,"allocated":100765696,"data_stored":111889747,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705765,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":128256,"kb_used_data":97336,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94243584,"statfs":{"total":96636764160,"available":96505430016,"internally_reserved":0,"allocated":99672064,"data_stored":108930298,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137312,"kb_used_data":110488,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94234528,"statfs":{"total":96636764160,"available":96496156672,"internally_reserved":0,"allocated":113139712,"data_stored":127962538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:40.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:40.355 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:40.350+0000 7f6926e60700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:40.358 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:40.354+0000 7f6926e60700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:40.390 INFO:tasks.rados.rados.0.smithi138.stdout:427: done (3 left) 2023-06-27T12:39:40.390 INFO:tasks.rados.rados.0.smithi138.stdout:428: done (2 left) 2023-06-27T12:39:40.390 INFO:tasks.rados.rados.0.smithi138.stdout:429: done (1 left) 2023-06-27T12:39:40.390 INFO:tasks.rados.rados.0.smithi138.stdout:431: done (0 left) 2023-06-27T12:39:40.391 INFO:tasks.rados.rados.0.smithi138.stdout:432: delete oid 12 current snap is 50 2023-06-27T12:39:40.463 INFO:tasks.rados.rados.0.smithi138.stdout:432: done (0 left) 2023-06-27T12:39:40.463 INFO:tasks.rados.rados.0.smithi138.stdout:433: write (excl) oid 43 current snap is 50 2023-06-27T12:39:40.463 INFO:tasks.rados.rados.0.smithi138.stdout:433: seq_num 222 ranges {531154=479515,1557826=443729,2713669=487328,3679295=51859} 2023-06-27T12:39:40.468 INFO:tasks.rados.rados.0.smithi138.stdout:433: writing smithi138138488-43 from 531154 to 1010669 tid 1 2023-06-27T12:39:40.473 INFO:tasks.rados.rados.0.smithi138.stdout:433: writing smithi138138488-43 from 1557826 to 2001555 tid 2 2023-06-27T12:39:40.480 INFO:tasks.rados.rados.0.smithi138.stdout:433: writing smithi138138488-43 from 2713669 to 3200997 tid 3 2023-06-27T12:39:40.483 INFO:tasks.rados.rados.0.smithi138.stdout:433: writing smithi138138488-43 from 3679295 to 3731154 tid 4 2023-06-27T12:39:40.483 INFO:tasks.rados.rados.0.smithi138.stdout:434: snap_create 2023-06-27T12:39:40.488 INFO:tasks.rados.rados.0.smithi138.stdout:433: finishing write tid 1 to smithi138138488-43 2023-06-27T12:39:40.538 INFO:tasks.rados.rados.0.smithi138.stdout:433: finishing write tid 2 to smithi138138488-43 2023-06-27T12:39:40.549 INFO:tasks.rados.rados.0.smithi138.stdout:433: finishing write tid 3 to smithi138138488-43 2023-06-27T12:39:40.551 INFO:tasks.rados.rados.0.smithi138.stdout:433: finishing write tid 4 to smithi138138488-43 2023-06-27T12:39:40.551 INFO:tasks.rados.rados.0.smithi138.stdout:433: finishing write tid 5 to smithi138138488-43 2023-06-27T12:39:40.552 INFO:tasks.rados.rados.0.smithi138.stdout:433: finishing write tid 6 to smithi138138488-43 2023-06-27T12:39:40.552 INFO:tasks.rados.rados.0.smithi138.stdout:433: oid 43 updating version 0 to 55 2023-06-27T12:39:40.553 INFO:tasks.rados.rados.0.smithi138.stdout:433: oid 43 version 55 is already newer than 55 2023-06-27T12:39:40.553 INFO:tasks.rados.rados.0.smithi138.stdout:433: oid 43 version 55 is already newer than 51 2023-06-27T12:39:40.553 INFO:tasks.rados.rados.0.smithi138.stdout:433: oid 43 version 55 is already newer than 55 2023-06-27T12:39:40.553 INFO:tasks.rados.rados.0.smithi138.stdout:433: oid 43 version 55 is already newer than 55 2023-06-27T12:39:40.553 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 55 (ObjNum 222 snap 50 seq_num 222) dirty exists 2023-06-27T12:39:40.553 INFO:tasks.rados.rados.0.smithi138.stdout:433: left oid 43 (ObjNum 222 snap 50 seq_num 222) 2023-06-27T12:39:41.397 INFO:tasks.rados.rados.0.smithi138.stdout:433: done (1 left) 2023-06-27T12:39:41.397 INFO:tasks.rados.rados.0.smithi138.stdout:434: done (0 left) 2023-06-27T12:39:41.397 INFO:tasks.rados.rados.0.smithi138.stdout:435: snap_create 2023-06-27T12:39:42.399 INFO:tasks.rados.rados.0.smithi138.stdout:435: done (0 left) 2023-06-27T12:39:42.399 INFO:tasks.rados.rados.0.smithi138.stdout:436: write oid 45 current snap is 52 2023-06-27T12:39:42.399 INFO:tasks.rados.rados.0.smithi138.stdout:436: seq_num 223 ranges {547961=554764,1616308=703740,2741092=698600} 2023-06-27T12:39:42.406 INFO:tasks.rados.rados.0.smithi138.stdout:436: writing smithi138138488-45 from 547961 to 1102725 tid 1 2023-06-27T12:39:42.415 INFO:tasks.rados.rados.0.smithi138.stdout:436: writing smithi138138488-45 from 1616308 to 2320048 tid 2 2023-06-27T12:39:42.422 INFO:tasks.rados.rados.0.smithi138.stdout:436: writing smithi138138488-45 from 2741092 to 3439692 tid 3 2023-06-27T12:39:42.423 INFO:tasks.rados.rados.0.smithi138.stdout:437: snap_create 2023-06-27T12:39:42.464 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:42.480 INFO:tasks.rados.rados.0.smithi138.stdout:436: finishing write tid 1 to smithi138138488-45 2023-06-27T12:39:42.483 INFO:tasks.rados.rados.0.smithi138.stdout:436: finishing write tid 2 to smithi138138488-45 2023-06-27T12:39:42.485 INFO:tasks.rados.rados.0.smithi138.stdout:436: finishing write tid 3 to smithi138138488-45 2023-06-27T12:39:42.486 INFO:tasks.rados.rados.0.smithi138.stdout:436: finishing write tid 4 to smithi138138488-45 2023-06-27T12:39:42.487 INFO:tasks.rados.rados.0.smithi138.stdout:436: finishing write tid 5 to smithi138138488-45 2023-06-27T12:39:42.488 INFO:tasks.rados.rados.0.smithi138.stdout:436: oid 45 updating version 0 to 150 2023-06-27T12:39:42.488 INFO:tasks.rados.rados.0.smithi138.stdout:436: oid 45 version 150 is already newer than 150 2023-06-27T12:39:42.488 INFO:tasks.rados.rados.0.smithi138.stdout:436: oid 45 version 150 is already newer than 150 2023-06-27T12:39:42.488 INFO:tasks.rados.rados.0.smithi138.stdout:436: oid 45 version 150 is already newer than 150 2023-06-27T12:39:42.488 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 150 (ObjNum 223 snap 52 seq_num 223) dirty exists 2023-06-27T12:39:42.488 INFO:tasks.rados.rados.0.smithi138.stdout:436: left oid 45 (ObjNum 223 snap 52 seq_num 223) 2023-06-27T12:39:42.595 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:42.589+0000 7f1cb0a32700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:42.597 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:42.594+0000 7f1cb0a32700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:43.405 INFO:tasks.rados.rados.0.smithi138.stdout:436: done (1 left) 2023-06-27T12:39:43.406 INFO:tasks.rados.rados.0.smithi138.stdout:437: done (0 left) 2023-06-27T12:39:43.406 INFO:tasks.rados.rados.0.smithi138.stdout:438: snap_create 2023-06-27T12:39:44.411 INFO:tasks.rados.rados.0.smithi138.stdout:438: done (0 left) 2023-06-27T12:39:44.411 INFO:tasks.rados.rados.0.smithi138.stdout:439: delete oid 13 current snap is 54 2023-06-27T12:39:44.517 INFO:tasks.rados.rados.0.smithi138.stdout:439: done (0 left) 2023-06-27T12:39:44.517 INFO:tasks.rados.rados.0.smithi138.stdout:440: write (excl) oid 36 current snap is 54 2023-06-27T12:39:44.518 INFO:tasks.rados.rados.0.smithi138.stdout:440: seq_num 225 ranges {581575=705262,1733272=707409,3195938=437497} 2023-06-27T12:39:44.524 INFO:tasks.rados.rados.0.smithi138.stdout:440: writing smithi138138488-36 from 581575 to 1286837 tid 1 2023-06-27T12:39:44.531 INFO:tasks.rados.rados.0.smithi138.stdout:440: writing smithi138138488-36 from 1733272 to 2440681 tid 2 2023-06-27T12:39:44.539 INFO:tasks.rados.rados.0.smithi138.stdout:440: writing smithi138138488-36 from 3195938 to 3633435 tid 3 2023-06-27T12:39:44.539 INFO:tasks.rados.rados.0.smithi138.stdout:441: snap_create 2023-06-27T12:39:44.599 INFO:tasks.rados.rados.0.smithi138.stdout:440: finishing write tid 1 to smithi138138488-36 2023-06-27T12:39:44.629 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:44.629 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:44.630 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:44.631 INFO:tasks.rados.rados.0.smithi138.stdout:440: finishing write tid 2 to smithi138138488-36 2023-06-27T12:39:44.631 INFO:tasks.rados.rados.0.smithi138.stdout:440: finishing write tid 3 to smithi138138488-36 2023-06-27T12:39:44.633 INFO:tasks.rados.rados.0.smithi138.stdout:440: finishing write tid 4 to smithi138138488-36 2023-06-27T12:39:44.633 INFO:tasks.rados.rados.0.smithi138.stdout:440: finishing write tid 5 to smithi138138488-36 2023-06-27T12:39:44.634 INFO:tasks.rados.rados.0.smithi138.stdout:440: oid 36 updating version 0 to 87 2023-06-27T12:39:44.634 INFO:tasks.rados.rados.0.smithi138.stdout:440: oid 36 version 87 is already newer than 83 2023-06-27T12:39:44.634 INFO:tasks.rados.rados.0.smithi138.stdout:440: oid 36 version 87 is already newer than 87 2023-06-27T12:39:44.634 INFO:tasks.rados.rados.0.smithi138.stdout:440: oid 36 version 87 is already newer than 87 2023-06-27T12:39:44.634 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 87 (ObjNum 225 snap 54 seq_num 225) dirty exists 2023-06-27T12:39:44.634 INFO:tasks.rados.rados.0.smithi138.stdout:440: left oid 36 (ObjNum 225 snap 54 seq_num 225) 2023-06-27T12:39:44.667 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:44.769 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:44.762+0000 7f5debeec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:44.772 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:44.769+0000 7f5debeec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:44.797 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:44.792+0000 7fef12f19700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:44.796+0000 7fef12f19700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:44.974 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:44.976 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:44.987 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-06-27T12:39:44.405870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113113068,"num_objects":154,"num_object_clones":99,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":769,"num_read_kb":449207,"num_write":914,"num_write_kb":228379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1064,"ondisk_log_size":1064,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":584756,"kb_used_data":461076,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376902604,"statfs":{"total":386547056640,"available":385948266496,"internally_reserved":0,"allocated":472141824,"data_stored":533947129,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1294139,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":11,"num_write_kb":3343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020730"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":209,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.942681+0000","last_change":"2023-06-27T12:39:39.805336+0000","last_active":"2023-06-27T12:39:41.942681+0000","last_peered":"2023-06-27T12:39:41.942681+0000","last_clean":"2023-06-27T12:39:41.942681+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:41.942681+0000","last_undegraded":"2023-06-27T12:39:41.942681+0000","last_fullsized":"2023-06-27T12:39:41.942681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"110'84","reported_seq":314,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.939698+0000","last_change":"2023-06-27T12:39:39.659986+0000","last_active":"2023-06-27T12:39:41.939698+0000","last_peered":"2023-06-27T12:39:41.939698+0000","last_clean":"2023-06-27T12:39:41.939698+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:41.939698+0000","last_undegraded":"2023-06-27T12:39:41.939698+0000","last_fullsized":"2023-06-27T12:39:41.939698+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9479148,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":49,"num_read_kb":43677,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"112'80","reported_seq":302,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-06-27T12:39:43.413251+0000","last_change":"2023-06-27T12:39:40.537919+0000","last_active":"2023-06-27T12:39:43.413251+0000","last_peered":"2023-06-27T12:39:43.413251+0000","last_clean":"2023-06-27T12:39:43.413251+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:43.413251+0000","last_undegraded":"2023-06-27T12:39:43.413251+0000","last_fullsized":"2023-06-27T12:39:43.413251+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9899323,"num_objects":12,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":48,"num_read_kb":20447,"num_write":71,"num_write_kb":20425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'40","reported_seq":258,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.936487+0000","last_change":"2023-06-27T12:39:39.805695+0000","last_active":"2023-06-27T12:39:41.936487+0000","last_peered":"2023-06-27T12:39:41.936487+0000","last_clean":"2023-06-27T12:39:41.936487+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:41.936487+0000","last_undegraded":"2023-06-27T12:39:41.936487+0000","last_fullsized":"2023-06-27T12:39:41.936487+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395047,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":38,"num_read_kb":13466,"num_write":35,"num_write_kb":10330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":245,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-06-27T12:39:42.421051+0000","last_change":"2023-06-27T12:39:39.950809+0000","last_active":"2023-06-27T12:39:42.421051+0000","last_peered":"2023-06-27T12:39:42.421051+0000","last_clean":"2023-06-27T12:39:42.421051+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:42.421051+0000","last_undegraded":"2023-06-27T12:39:42.421051+0000","last_fullsized":"2023-06-27T12:39:42.421051+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":21,"num_read_kb":19187,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"110'112","reported_seq":336,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.939024+0000","last_change":"2023-06-27T12:39:39.807303+0000","last_active":"2023-06-27T12:39:41.939024+0000","last_peered":"2023-06-27T12:39:41.939024+0000","last_clean":"2023-06-27T12:39:41.939024+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:41.939024+0000","last_undegraded":"2023-06-27T12:39:41.939024+0000","last_fullsized":"2023-06-27T12:39:41.939024+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17243949,"num_objects":18,"num_object_clones":12,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":116,"num_read_kb":76831,"num_write":87,"num_write_kb":25064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":280,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.932966+0000","last_change":"2023-06-27T12:39:39.806117+0000","last_active":"2023-06-27T12:39:41.932966+0000","last_peered":"2023-06-27T12:39:41.932966+0000","last_clean":"2023-06-27T12:39:41.932966+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:41.932966+0000","last_undegraded":"2023-06-27T12:39:41.932966+0000","last_fullsized":"2023-06-27T12:39:41.932966+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":24452,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"100'23","reported_seq":246,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-06-27T12:39:43.416500+0000","last_change":"2023-06-27T12:39:40.409974+0000","last_active":"2023-06-27T12:39:43.416500+0000","last_peered":"2023-06-27T12:39:43.416500+0000","last_clean":"2023-06-27T12:39:43.416500+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:43.416500+0000","last_undegraded":"2023-06-27T12:39:43.416500+0000","last_fullsized":"2023-06-27T12:39:43.416500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3479049,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":25159,"num_write":18,"num_write_kb":5463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":177,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.935675+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:41.935675+0000","last_peered":"2023-06-27T12:39:41.935675+0000","last_clean":"2023-06-27T12:39:41.935675+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:41.935675+0000","last_undegraded":"2023-06-27T12:39:41.935675+0000","last_fullsized":"2023-06-27T12:39:41.935675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":212,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-06-27T12:39:43.410389+0000","last_change":"2023-06-27T12:39:40.409126+0000","last_active":"2023-06-27T12:39:43.410389+0000","last_peered":"2023-06-27T12:39:43.410389+0000","last_clean":"2023-06-27T12:39:43.410389+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:43.410389+0000","last_undegraded":"2023-06-27T12:39:43.410389+0000","last_fullsized":"2023-06-27T12:39:43.410389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":165,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.941094+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:41.941094+0000","last_peered":"2023-06-27T12:39:41.941094+0000","last_clean":"2023-06-27T12:39:41.941094+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:41.941094+0000","last_undegraded":"2023-06-27T12:39:41.941094+0000","last_fullsized":"2023-06-27T12:39:41.941094+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"100'51","reported_seq":270,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.937208+0000","last_change":"2023-06-27T12:39:39.647421+0000","last_active":"2023-06-27T12:39:41.937208+0000","last_peered":"2023-06-27T12:39:41.937208+0000","last_clean":"2023-06-27T12:39:41.937208+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:41.937208+0000","last_undegraded":"2023-06-27T12:39:41.937208+0000","last_fullsized":"2023-06-27T12:39:41.937208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168169,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":28,"num_read_kb":20337,"num_write":38,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-06-27T12:39:43.414708+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:43.414708+0000","last_peered":"2023-06-27T12:39:43.414708+0000","last_clean":"2023-06-27T12:39:43.414708+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:43.414708+0000","last_undegraded":"2023-06-27T12:39:43.414708+0000","last_fullsized":"2023-06-27T12:39:43.414708+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"114'150","reported_seq":355,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-06-27T12:39:42.455493+0000","last_change":"2023-06-27T12:39:39.890684+0000","last_active":"2023-06-27T12:39:42.455493+0000","last_peered":"2023-06-27T12:39:42.455493+0000","last_clean":"2023-06-27T12:39:42.455493+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:42.455493+0000","last_undegraded":"2023-06-27T12:39:42.455493+0000","last_fullsized":"2023-06-27T12:39:42.455493+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10269077,"num_objects":19,"num_object_clones":14,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":72,"num_read_kb":62731,"num_write":110,"num_write_kb":33131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-06-27T12:39:43.412715+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:43.412715+0000","last_peered":"2023-06-27T12:39:43.412715+0000","last_clean":"2023-06-27T12:39:43.412715+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:43.412715+0000","last_undegraded":"2023-06-27T12:39:43.412715+0000","last_fullsized":"2023-06-27T12:39:43.412715+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":298,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-06-27T12:39:42.420283+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:42.420283+0000","last_peered":"2023-06-27T12:39:42.420283+0000","last_clean":"2023-06-27T12:39:42.420283+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:42.420283+0000","last_undegraded":"2023-06-27T12:39:42.420283+0000","last_fullsized":"2023-06-27T12:39:42.420283+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"110'112","reported_seq":349,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-06-27T12:39:43.415594+0000","last_change":"2023-06-27T12:39:40.409109+0000","last_active":"2023-06-27T12:39:43.415594+0000","last_peered":"2023-06-27T12:39:43.415594+0000","last_clean":"2023-06-27T12:39:43.415594+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:43.415594+0000","last_undegraded":"2023-06-27T12:39:43.415594+0000","last_fullsized":"2023-06-27T12:39:43.415594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15697672,"num_objects":18,"num_object_clones":12,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":96,"num_read_kb":38300,"num_write":98,"num_write_kb":28357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":165,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.933208+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:41.933208+0000","last_peered":"2023-06-27T12:39:41.933208+0000","last_clean":"2023-06-27T12:39:41.933208+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:41.933208+0000","last_undegraded":"2023-06-27T12:39:41.933208+0000","last_fullsized":"2023-06-27T12:39:41.933208+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"102'51","reported_seq":263,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-06-27T12:39:42.420103+0000","last_change":"2023-06-27T12:39:39.888834+0000","last_active":"2023-06-27T12:39:42.420103+0000","last_peered":"2023-06-27T12:39:42.420103+0000","last_clean":"2023-06-27T12:39:42.420103+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:42.420103+0000","last_undegraded":"2023-06-27T12:39:42.420103+0000","last_fullsized":"2023-06-27T12:39:42.420103+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":27,"num_read_kb":24672,"num_write":40,"num_write_kb":11607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-06-27T12:39:42.422197+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:42.422197+0000","last_peered":"2023-06-27T12:39:42.422197+0000","last_clean":"2023-06-27T12:39:42.422197+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:42.422197+0000","last_undegraded":"2023-06-27T12:39:42.422197+0000","last_fullsized":"2023-06-27T12:39:42.422197+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'55","reported_seq":288,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.934883+0000","last_change":"2023-06-27T12:39:39.636559+0000","last_active":"2023-06-27T12:39:41.934883+0000","last_peered":"2023-06-27T12:39:41.934883+0000","last_clean":"2023-06-27T12:39:41.934883+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:41.934883+0000","last_undegraded":"2023-06-27T12:39:41.934883+0000","last_fullsized":"2023-06-27T12:39:41.934883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185042,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":35971,"num_write":52,"num_write_kb":15517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"100'41","reported_seq":248,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-06-27T12:39:42.420950+0000","last_change":"2023-06-27T12:39:39.889733+0000","last_active":"2023-06-27T12:39:42.420950+0000","last_peered":"2023-06-27T12:39:42.420950+0000","last_clean":"2023-06-27T12:39:42.420950+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:42.420950+0000","last_undegraded":"2023-06-27T12:39:42.420950+0000","last_fullsized":"2023-06-27T12:39:42.420950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285752,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":17,"num_read_kb":13728,"num_write":30,"num_write_kb":8133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.936179+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:41.936179+0000","last_peered":"2023-06-27T12:39:41.936179+0000","last_clean":"2023-06-27T12:39:41.936179+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:41.936179+0000","last_undegraded":"2023-06-27T12:39:41.936179+0000","last_fullsized":"2023-06-27T12:39:41.936179+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":173,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-06-27T12:39:43.412543+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:43.412543+0000","last_peered":"2023-06-27T12:39:43.412543+0000","last_clean":"2023-06-27T12:39:43.412543+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:43.412543+0000","last_undegraded":"2023-06-27T12:39:43.412543+0000","last_fullsized":"2023-06-27T12:39:43.412543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"107'60","reported_seq":281,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-06-27T12:39:41.942853+0000","last_change":"2023-06-27T12:39:39.658015+0000","last_active":"2023-06-27T12:39:41.942853+0000","last_peered":"2023-06-27T12:39:41.942853+0000","last_clean":"2023-06-27T12:39:41.942853+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:41.942853+0000","last_undegraded":"2023-06-27T12:39:41.942853+0000","last_fullsized":"2023-06-27T12:39:41.942853+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9277464,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30157,"num_write":45,"num_write_kb":13422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":112522700,"num_objects":152,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":666,"num_read_kb":449116,"num_write":777,"num_write_kb":226485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":967,"ondisk_log_size":967,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":152992,"kb_used_data":122072,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94218848,"statfs":{"total":96636764160,"available":96480100352,"internally_reserved":0,"allocated":125001728,"data_stored":140551130,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":139668,"kb_used_data":108748,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94232172,"statfs":{"total":96636764160,"available":96493744128,"internally_reserved":0,"allocated":111357952,"data_stored":126275320,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":136500,"kb_used_data":105580,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94235340,"statfs":{"total":96636764160,"available":96496988160,"internally_reserved":0,"allocated":108113920,"data_stored":119629670,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155596,"kb_used_data":124676,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94216244,"statfs":{"total":96636764160,"available":96477433856,"internally_reserved":0,"allocated":127668224,"data_stored":147491009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:45.422 INFO:tasks.rados.rados.0.smithi138.stdout:440: done (1 left) 2023-06-27T12:39:45.423 INFO:tasks.rados.rados.0.smithi138.stdout:441: done (0 left) 2023-06-27T12:39:45.423 INFO:tasks.rados.rados.0.smithi138.stdout:442: read oid 48 snap -1 2023-06-27T12:39:45.423 INFO:tasks.rados.rados.0.smithi138.stdout:442: expect (ObjNum 216 snap 47 seq_num 216) 2023-06-27T12:39:45.423 INFO:tasks.rados.rados.0.smithi138.stdout:443: read oid 43 snap 42 2023-06-27T12:39:45.423 INFO:tasks.rados.rados.0.smithi138.stdout:443: expect deleted 2023-06-27T12:39:45.423 INFO:tasks.rados.rados.0.smithi138.stdout:444: write oid 50 current snap is 55 2023-06-27T12:39:45.423 INFO:tasks.rados.rados.0.smithi138.stdout:444: seq_num 226 ranges {598382=780511,2075401=451067,3107008=648769} 2023-06-27T12:39:45.431 INFO:tasks.rados.rados.0.smithi138.stdout:444: writing smithi138138488-50 from 598382 to 1378893 tid 1 2023-06-27T12:39:45.439 INFO:tasks.rados.rados.0.smithi138.stdout:444: writing smithi138138488-50 from 2075401 to 2526468 tid 2 2023-06-27T12:39:45.449 INFO:tasks.rados.rados.0.smithi138.stdout:444: writing smithi138138488-50 from 3107008 to 3755777 tid 3 2023-06-27T12:39:45.449 INFO:tasks.rados.rados.0.smithi138.stdout:445: read oid 1 snap -1 2023-06-27T12:39:45.450 INFO:tasks.rados.rados.0.smithi138.stdout:445: expect (ObjNum 96 snap 7 seq_num 96) 2023-06-27T12:39:45.450 INFO:tasks.rados.rados.0.smithi138.stdout:446: write oid 29 current snap is 55 2023-06-27T12:39:45.450 INFO:tasks.rados.rados.0.smithi138.stdout:446: seq_num 227 ranges {615189=455760,1733883=711078,3134431=460041} 2023-06-27T12:39:45.456 INFO:tasks.rados.rados.0.smithi138.stdout:446: writing smithi138138488-29 from 615189 to 1070949 tid 1 2023-06-27T12:39:45.468 INFO:tasks.rados.rados.0.smithi138.stdout:446: writing smithi138138488-29 from 1733883 to 2444961 tid 2 2023-06-27T12:39:45.475 INFO:tasks.rados.rados.0.smithi138.stdout:446: writing smithi138138488-29 from 3134431 to 3594472 tid 3 2023-06-27T12:39:45.476 INFO:tasks.rados.rados.0.smithi138.stdout:447: snap_remove snap 49 2023-06-27T12:39:46.427 INFO:tasks.rados.rados.0.smithi138.stdout:447: done (5 left) 2023-06-27T12:39:46.428 INFO:tasks.rados.rados.0.smithi138.stdout:448: delete oid 12 current snap is 55 2023-06-27T12:39:46.428 INFO:tasks.rados.rados.0.smithi138.stdout:442: expect (ObjNum 216 snap 47 seq_num 216) 2023-06-27T12:39:46.618 INFO:tasks.rados.rados.0.smithi138.stdout:445: expect (ObjNum 96 snap 7 seq_num 96) 2023-06-27T12:39:46.836 INFO:tasks.rados.rados.0.smithi138.stdout:446: finishing write tid 1 to smithi138138488-29 2023-06-27T12:39:46.837 INFO:tasks.rados.rados.0.smithi138.stdout:444: finishing write tid 1 to smithi138138488-50 2023-06-27T12:39:46.837 INFO:tasks.rados.rados.0.smithi138.stdout:444: finishing write tid 2 to smithi138138488-50 2023-06-27T12:39:46.837 INFO:tasks.rados.rados.0.smithi138.stdout:444: finishing write tid 3 to smithi138138488-50 2023-06-27T12:39:46.837 INFO:tasks.rados.rados.0.smithi138.stdout:444: finishing write tid 4 to smithi138138488-50 2023-06-27T12:39:46.837 INFO:tasks.rados.rados.0.smithi138.stdout:444: finishing write tid 5 to smithi138138488-50 2023-06-27T12:39:46.838 INFO:tasks.rados.rados.0.smithi138.stdout:444: oid 50 updating version 0 to 32 2023-06-27T12:39:46.838 INFO:tasks.rados.rados.0.smithi138.stdout:444: oid 50 version 32 is already newer than 32 2023-06-27T12:39:46.838 INFO:tasks.rados.rados.0.smithi138.stdout:444: oid 50 version 32 is already newer than 32 2023-06-27T12:39:46.838 INFO:tasks.rados.rados.0.smithi138.stdout:444: oid 50 version 32 is already newer than 32 2023-06-27T12:39:46.838 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 32 (ObjNum 226 snap 55 seq_num 226) dirty exists 2023-06-27T12:39:46.838 INFO:tasks.rados.rados.0.smithi138.stdout:444: left oid 50 (ObjNum 226 snap 55 seq_num 226) 2023-06-27T12:39:46.838 INFO:tasks.rados.rados.0.smithi138.stdout:446: finishing write tid 2 to smithi138138488-29 2023-06-27T12:39:46.838 INFO:tasks.rados.rados.0.smithi138.stdout:446: finishing write tid 3 to smithi138138488-29 2023-06-27T12:39:46.839 INFO:tasks.rados.rados.0.smithi138.stdout:446: finishing write tid 4 to smithi138138488-29 2023-06-27T12:39:46.839 INFO:tasks.rados.rados.0.smithi138.stdout:446: finishing write tid 5 to smithi138138488-29 2023-06-27T12:39:46.839 INFO:tasks.rados.rados.0.smithi138.stdout:446: oid 29 updating version 0 to 49 2023-06-27T12:39:46.839 INFO:tasks.rados.rados.0.smithi138.stdout:446: oid 29 version 49 is already newer than 49 2023-06-27T12:39:46.839 INFO:tasks.rados.rados.0.smithi138.stdout:446: oid 29 version 49 is already newer than 49 2023-06-27T12:39:46.839 INFO:tasks.rados.rados.0.smithi138.stdout:446: oid 29 version 49 is already newer than 49 2023-06-27T12:39:46.839 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 49 (ObjNum 227 snap 55 seq_num 227) dirty exists 2023-06-27T12:39:46.839 INFO:tasks.rados.rados.0.smithi138.stdout:446: left oid 29 (ObjNum 227 snap 55 seq_num 227) 2023-06-27T12:39:46.840 INFO:tasks.rados.rados.0.smithi138.stdout:442: done (5 left) 2023-06-27T12:39:46.840 INFO:tasks.rados.rados.0.smithi138.stdout:443: done (4 left) 2023-06-27T12:39:46.840 INFO:tasks.rados.rados.0.smithi138.stdout:444: done (3 left) 2023-06-27T12:39:46.840 INFO:tasks.rados.rados.0.smithi138.stdout:445: done (2 left) 2023-06-27T12:39:46.840 INFO:tasks.rados.rados.0.smithi138.stdout:446: done (1 left) 2023-06-27T12:39:46.840 INFO:tasks.rados.rados.0.smithi138.stdout:448: done (0 left) 2023-06-27T12:39:46.840 INFO:tasks.rados.rados.0.smithi138.stdout:449: read oid 29 snap 47 2023-06-27T12:39:46.841 INFO:tasks.rados.rados.0.smithi138.stdout:449: expect deleted 2023-06-27T12:39:46.841 INFO:tasks.rados.rados.0.smithi138.stdout:450: read oid 37 snap -1 2023-06-27T12:39:46.841 INFO:tasks.rados.rados.0.smithi138.stdout:450: expect deleted 2023-06-27T12:39:46.841 INFO:tasks.rados.rados.0.smithi138.stdout:451: write (excl) oid 18 current snap is 55 2023-06-27T12:39:46.841 INFO:tasks.rados.rados.0.smithi138.stdout:451: seq_num 229 ranges {648803=722611,1967200=714747,3421983=426820} 2023-06-27T12:39:46.843 INFO:tasks.rados.rados.0.smithi138.stdout:451: writing smithi138138488-18 from 648803 to 1371414 tid 1 2023-06-27T12:39:46.850 INFO:tasks.rados.rados.0.smithi138.stdout:451: writing smithi138138488-18 from 1967200 to 2681947 tid 2 2023-06-27T12:39:46.857 INFO:tasks.rados.rados.0.smithi138.stdout:451: writing smithi138138488-18 from 3421983 to 3848803 tid 3 2023-06-27T12:39:46.858 INFO:tasks.rados.rados.0.smithi138.stdout:452: read oid 44 snap 47 2023-06-27T12:39:46.858 INFO:tasks.rados.rados.0.smithi138.stdout:452: expect (ObjNum 105 snap 9 seq_num 105) 2023-06-27T12:39:46.858 INFO:tasks.rados.rados.0.smithi138.stdout:453: write (excl) oid 39 current snap is 55 2023-06-27T12:39:46.858 INFO:tasks.rados.rados.0.smithi138.stdout:453: seq_num 230 ranges {665610=797860,1909329=458405,2816700=410210} 2023-06-27T12:39:46.866 INFO:tasks.rados.rados.0.smithi138.stdout:453: writing smithi138138488-39 from 665610 to 1463470 tid 1 2023-06-27T12:39:46.871 INFO:tasks.rados.rados.0.smithi138.stdout:453: writing smithi138138488-39 from 1909329 to 2367734 tid 2 2023-06-27T12:39:46.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:46.876 INFO:tasks.rados.rados.0.smithi138.stdout:453: writing smithi138138488-39 from 2816700 to 3226910 tid 3 2023-06-27T12:39:46.876 INFO:tasks.rados.rados.0.smithi138.stdout:454: write oid 13 current snap is 55 2023-06-27T12:39:46.877 INFO:tasks.rados.rados.0.smithi138.stdout:454: seq_num 231 ranges {682417=473109,1567811=602063,2844123=621482} 2023-06-27T12:39:46.884 INFO:tasks.rados.rados.0.smithi138.stdout:454: writing smithi138138488-13 from 682417 to 1155526 tid 1 2023-06-27T12:39:46.891 INFO:tasks.rados.rados.0.smithi138.stdout:454: writing smithi138138488-13 from 1567811 to 2169874 tid 2 2023-06-27T12:39:46.899 INFO:tasks.rados.rados.0.smithi138.stdout:454: writing smithi138138488-13 from 2844123 to 3465605 tid 3 2023-06-27T12:39:46.899 INFO:tasks.rados.rados.0.smithi138.stdout:455: snap_create 2023-06-27T12:39:46.899 INFO:tasks.rados.rados.0.smithi138.stdout:452: expect (ObjNum 105 snap 9 seq_num 105) 2023-06-27T12:39:47.010 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:47.005+0000 7fed7a12d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:47.013 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:47.009+0000 7fed7a12d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:47.139 INFO:tasks.rados.rados.0.smithi138.stdout:451: finishing write tid 1 to smithi138138488-18 2023-06-27T12:39:47.139 INFO:tasks.rados.rados.0.smithi138.stdout:451: finishing write tid 2 to smithi138138488-18 2023-06-27T12:39:47.139 INFO:tasks.rados.rados.0.smithi138.stdout:453: finishing write tid 1 to smithi138138488-39 2023-06-27T12:39:47.139 INFO:tasks.rados.rados.0.smithi138.stdout:451: finishing write tid 3 to smithi138138488-18 2023-06-27T12:39:47.140 INFO:tasks.rados.rados.0.smithi138.stdout:451: finishing write tid 4 to smithi138138488-18 2023-06-27T12:39:47.140 INFO:tasks.rados.rados.0.smithi138.stdout:451: finishing write tid 5 to smithi138138488-18 2023-06-27T12:39:47.140 INFO:tasks.rados.rados.0.smithi138.stdout:451: oid 18 updating version 0 to 97 2023-06-27T12:39:47.140 INFO:tasks.rados.rados.0.smithi138.stdout:451: oid 18 version 97 is already newer than 93 2023-06-27T12:39:47.140 INFO:tasks.rados.rados.0.smithi138.stdout:451: oid 18 version 97 is already newer than 97 2023-06-27T12:39:47.140 INFO:tasks.rados.rados.0.smithi138.stdout:451: oid 18 version 97 is already newer than 97 2023-06-27T12:39:47.140 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 97 (ObjNum 229 snap 55 seq_num 229) dirty exists 2023-06-27T12:39:47.141 INFO:tasks.rados.rados.0.smithi138.stdout:451: left oid 18 (ObjNum 229 snap 55 seq_num 229) 2023-06-27T12:39:47.141 INFO:tasks.rados.rados.0.smithi138.stdout:454: finishing write tid 1 to smithi138138488-13 2023-06-27T12:39:47.141 INFO:tasks.rados.rados.0.smithi138.stdout:454: finishing write tid 2 to smithi138138488-13 2023-06-27T12:39:47.141 INFO:tasks.rados.rados.0.smithi138.stdout:454: finishing write tid 3 to smithi138138488-13 2023-06-27T12:39:47.141 INFO:tasks.rados.rados.0.smithi138.stdout:454: finishing write tid 4 to smithi138138488-13 2023-06-27T12:39:47.141 INFO:tasks.rados.rados.0.smithi138.stdout:454: finishing write tid 5 to smithi138138488-13 2023-06-27T12:39:47.141 INFO:tasks.rados.rados.0.smithi138.stdout:454: oid 13 updating version 0 to 160 2023-06-27T12:39:47.142 INFO:tasks.rados.rados.0.smithi138.stdout:454: oid 13 version 160 is already newer than 160 2023-06-27T12:39:47.142 INFO:tasks.rados.rados.0.smithi138.stdout:454: oid 13 version 160 is already newer than 160 2023-06-27T12:39:47.142 INFO:tasks.rados.rados.0.smithi138.stdout:454: oid 13 version 160 is already newer than 160 2023-06-27T12:39:47.142 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 160 (ObjNum 231 snap 55 seq_num 231) dirty exists 2023-06-27T12:39:47.142 INFO:tasks.rados.rados.0.smithi138.stdout:454: left oid 13 (ObjNum 231 snap 55 seq_num 231) 2023-06-27T12:39:47.142 INFO:tasks.rados.rados.0.smithi138.stdout:453: finishing write tid 2 to smithi138138488-39 2023-06-27T12:39:47.142 INFO:tasks.rados.rados.0.smithi138.stdout:453: finishing write tid 3 to smithi138138488-39 2023-06-27T12:39:47.143 INFO:tasks.rados.rados.0.smithi138.stdout:453: finishing write tid 4 to smithi138138488-39 2023-06-27T12:39:47.143 INFO:tasks.rados.rados.0.smithi138.stdout:453: finishing write tid 5 to smithi138138488-39 2023-06-27T12:39:47.143 INFO:tasks.rados.rados.0.smithi138.stdout:453: oid 39 updating version 0 to 61 2023-06-27T12:39:47.143 INFO:tasks.rados.rados.0.smithi138.stdout:453: oid 39 version 61 is already newer than 61 2023-06-27T12:39:47.143 INFO:tasks.rados.rados.0.smithi138.stdout:453: oid 39 version 61 is already newer than 58 2023-06-27T12:39:47.143 INFO:tasks.rados.rados.0.smithi138.stdout:453: oid 39 version 61 is already newer than 61 2023-06-27T12:39:47.143 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 39 v 61 (ObjNum 230 snap 55 seq_num 230) dirty exists 2023-06-27T12:39:47.143 INFO:tasks.rados.rados.0.smithi138.stdout:453: left oid 39 (ObjNum 230 snap 55 seq_num 230) 2023-06-27T12:39:47.430 INFO:tasks.rados.rados.0.smithi138.stdout:449: done (6 left) 2023-06-27T12:39:47.430 INFO:tasks.rados.rados.0.smithi138.stdout:450: done (5 left) 2023-06-27T12:39:47.430 INFO:tasks.rados.rados.0.smithi138.stdout:451: done (4 left) 2023-06-27T12:39:47.430 INFO:tasks.rados.rados.0.smithi138.stdout:452: done (3 left) 2023-06-27T12:39:47.430 INFO:tasks.rados.rados.0.smithi138.stdout:453: done (2 left) 2023-06-27T12:39:47.430 INFO:tasks.rados.rados.0.smithi138.stdout:454: done (1 left) 2023-06-27T12:39:47.431 INFO:tasks.rados.rados.0.smithi138.stdout:455: done (0 left) 2023-06-27T12:39:47.431 INFO:tasks.rados.rados.0.smithi138.stdout:456: read oid 33 snap 55 2023-06-27T12:39:47.431 INFO:tasks.rados.rados.0.smithi138.stdout:456: expect (ObjNum 199 snap 41 seq_num 199) 2023-06-27T12:39:47.431 INFO:tasks.rados.rados.0.smithi138.stdout:457: snap_create 2023-06-27T12:39:47.462 INFO:tasks.rados.rados.0.smithi138.stdout:456: expect (ObjNum 199 snap 41 seq_num 199) 2023-06-27T12:39:48.439 INFO:tasks.rados.rados.0.smithi138.stdout:456: done (1 left) 2023-06-27T12:39:48.439 INFO:tasks.rados.rados.0.smithi138.stdout:457: done (0 left) 2023-06-27T12:39:48.439 INFO:tasks.rados.rados.0.smithi138.stdout:458: write (excl) oid 45 current snap is 57 2023-06-27T12:39:48.440 INFO:tasks.rados.rados.0.smithi138.stdout:458: seq_num 232 ranges {699224=548358,2026293=462074,3271546=432754} 2023-06-27T12:39:48.447 INFO:tasks.rados.rados.0.smithi138.stdout:458: writing smithi138138488-45 from 699224 to 1247582 tid 1 2023-06-27T12:39:48.455 INFO:tasks.rados.rados.0.smithi138.stdout:458: writing smithi138138488-45 from 2026293 to 2488367 tid 2 2023-06-27T12:39:48.462 INFO:tasks.rados.rados.0.smithi138.stdout:458: writing smithi138138488-45 from 3271546 to 3704300 tid 3 2023-06-27T12:39:48.462 INFO:tasks.rados.rados.0.smithi138.stdout:459: snap_create 2023-06-27T12:39:48.522 INFO:tasks.rados.rados.0.smithi138.stdout:458: finishing write tid 1 to smithi138138488-45 2023-06-27T12:39:48.558 INFO:tasks.rados.rados.0.smithi138.stdout:458: finishing write tid 2 to smithi138138488-45 2023-06-27T12:39:48.611 INFO:tasks.rados.rados.0.smithi138.stdout:458: finishing write tid 3 to smithi138138488-45 2023-06-27T12:39:48.611 INFO:tasks.rados.rados.0.smithi138.stdout:458: finishing write tid 4 to smithi138138488-45 2023-06-27T12:39:48.613 INFO:tasks.rados.rados.0.smithi138.stdout:458: finishing write tid 5 to smithi138138488-45 2023-06-27T12:39:48.613 INFO:tasks.rados.rados.0.smithi138.stdout:458: oid 45 updating version 0 to 161 2023-06-27T12:39:48.613 INFO:tasks.rados.rados.0.smithi138.stdout:458: oid 45 updating version 161 to 165 2023-06-27T12:39:48.613 INFO:tasks.rados.rados.0.smithi138.stdout:458: oid 45 version 165 is already newer than 165 2023-06-27T12:39:48.613 INFO:tasks.rados.rados.0.smithi138.stdout:458: oid 45 version 165 is already newer than 165 2023-06-27T12:39:48.614 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 165 (ObjNum 232 snap 57 seq_num 232) dirty exists 2023-06-27T12:39:48.614 INFO:tasks.rados.rados.0.smithi138.stdout:458: left oid 45 (ObjNum 232 snap 57 seq_num 232) 2023-06-27T12:39:49.083 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:49.210 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:49.205+0000 7f2914938700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:49.213 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:49.209+0000 7f2914938700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:49.443 INFO:tasks.rados.rados.0.smithi138.stdout:458: done (1 left) 2023-06-27T12:39:49.443 INFO:tasks.rados.rados.0.smithi138.stdout:459: done (0 left) 2023-06-27T12:39:49.444 INFO:tasks.rados.rados.0.smithi138.stdout:460: read oid 35 snap 52 2023-06-27T12:39:49.444 INFO:tasks.rados.rados.0.smithi138.stdout:460: expect deleted 2023-06-27T12:39:49.444 INFO:tasks.rados.rados.0.smithi138.stdout:461: read oid 2 snap -1 2023-06-27T12:39:49.444 INFO:tasks.rados.rados.0.smithi138.stdout:461: expect deleted 2023-06-27T12:39:49.444 INFO:tasks.rados.rados.0.smithi138.stdout:462: snap_remove snap 48 2023-06-27T12:39:49.992 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:49.992 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:49.992 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:50.113+0000 7fa759fce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:50.123 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:50.120+0000 7fa759fce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:50.317 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:50.319 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:50.330 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2023-06-27T12:39:49.927619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":122861128,"num_objects":160,"num_object_clones":104,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":805,"num_read_kb":483951,"num_write":949,"num_write_kb":240135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1123,"ondisk_log_size":1123,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":620328,"kb_used_data":496648,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376867032,"statfs":{"total":386547056640,"available":385911840768,"internally_reserved":0,"allocated":508567552,"data_stored":581597293,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9748060,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":36,"num_read_kb":34744,"num_write":35,"num_write_kb":11756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000667"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":218,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.447338+0000","last_change":"2023-06-27T12:39:46.443042+0000","last_active":"2023-06-27T12:39:48.447338+0000","last_peered":"2023-06-27T12:39:48.447338+0000","last_clean":"2023-06-27T12:39:48.447338+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:48.447338+0000","last_undegraded":"2023-06-27T12:39:48.447338+0000","last_fullsized":"2023-06-27T12:39:48.447338+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"110'84","reported_seq":323,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.559967+0000","last_change":"2023-06-27T12:39:46.848524+0000","last_active":"2023-06-27T12:39:48.559967+0000","last_peered":"2023-06-27T12:39:48.559967+0000","last_clean":"2023-06-27T12:39:48.559967+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:48.559967+0000","last_undegraded":"2023-06-27T12:39:48.559967+0000","last_fullsized":"2023-06-27T12:39:48.559967+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9479148,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":49,"num_read_kb":43677,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"118'97","reported_seq":322,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.556932+0000","last_change":"2023-06-27T12:39:46.481632+0000","last_active":"2023-06-27T12:39:48.556932+0000","last_peered":"2023-06-27T12:39:48.556932+0000","last_clean":"2023-06-27T12:39:48.556932+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:48.556932+0000","last_undegraded":"2023-06-27T12:39:48.556932+0000","last_fullsized":"2023-06-27T12:39:48.556932+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11227076,"num_objects":14,"num_object_clones":9,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":20449,"num_write":81,"num_write_kb":24054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'40","reported_seq":270,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.439084+0000","last_change":"2023-06-27T12:39:46.449108+0000","last_active":"2023-06-27T12:39:48.439084+0000","last_peered":"2023-06-27T12:39:48.439084+0000","last_clean":"2023-06-27T12:39:48.439084+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:48.439084+0000","last_undegraded":"2023-06-27T12:39:48.439084+0000","last_fullsized":"2023-06-27T12:39:48.439084+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395047,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":45,"num_read_kb":21985,"num_write":35,"num_write_kb":10330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":256,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.465400+0000","last_change":"2023-06-27T12:39:46.841228+0000","last_active":"2023-06-27T12:39:48.465400+0000","last_peered":"2023-06-27T12:39:48.465400+0000","last_clean":"2023-06-27T12:39:48.465400+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:48.465400+0000","last_undegraded":"2023-06-27T12:39:48.465400+0000","last_fullsized":"2023-06-27T12:39:48.465400+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":21,"num_read_kb":19187,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"110'112","reported_seq":348,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.445325+0000","last_change":"2023-06-27T12:39:46.444223+0000","last_active":"2023-06-27T12:39:48.445325+0000","last_peered":"2023-06-27T12:39:48.445325+0000","last_clean":"2023-06-27T12:39:48.445325+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:48.445325+0000","last_undegraded":"2023-06-27T12:39:48.445325+0000","last_fullsized":"2023-06-27T12:39:48.445325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17243949,"num_objects":18,"num_object_clones":12,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":123,"num_read_kb":87862,"num_write":87,"num_write_kb":25064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":289,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.442009+0000","last_change":"2023-06-27T12:39:46.448633+0000","last_active":"2023-06-27T12:39:48.442009+0000","last_peered":"2023-06-27T12:39:48.442009+0000","last_clean":"2023-06-27T12:39:48.442009+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:48.442009+0000","last_undegraded":"2023-06-27T12:39:48.442009+0000","last_fullsized":"2023-06-27T12:39:48.442009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":24452,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"117'32","reported_seq":260,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.556697+0000","last_change":"2023-06-27T12:39:46.506286+0000","last_active":"2023-06-27T12:39:48.556697+0000","last_peered":"2023-06-27T12:39:48.556697+0000","last_clean":"2023-06-27T12:39:48.556697+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:48.556697+0000","last_undegraded":"2023-06-27T12:39:48.556697+0000","last_fullsized":"2023-06-27T12:39:48.556697+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3798382,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25160,"num_write":23,"num_write_kb":7301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":185,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.551425+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:48.551425+0000","last_peered":"2023-06-27T12:39:48.551425+0000","last_clean":"2023-06-27T12:39:48.551425+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:48.551425+0000","last_undegraded":"2023-06-27T12:39:48.551425+0000","last_fullsized":"2023-06-27T12:39:48.551425+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":221,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.552032+0000","last_change":"2023-06-27T12:39:46.459883+0000","last_active":"2023-06-27T12:39:48.552032+0000","last_peered":"2023-06-27T12:39:48.552032+0000","last_clean":"2023-06-27T12:39:48.552032+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:48.552032+0000","last_undegraded":"2023-06-27T12:39:48.552032+0000","last_fullsized":"2023-06-27T12:39:48.552032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.450318+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:48.450318+0000","last_peered":"2023-06-27T12:39:48.450318+0000","last_clean":"2023-06-27T12:39:48.450318+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:48.450318+0000","last_undegraded":"2023-06-27T12:39:48.450318+0000","last_fullsized":"2023-06-27T12:39:48.450318+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"100'51","reported_seq":279,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.560696+0000","last_change":"2023-06-27T12:39:46.848229+0000","last_active":"2023-06-27T12:39:48.560696+0000","last_peered":"2023-06-27T12:39:48.560696+0000","last_clean":"2023-06-27T12:39:48.560696+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:48.560696+0000","last_undegraded":"2023-06-27T12:39:48.560696+0000","last_fullsized":"2023-06-27T12:39:48.560696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168169,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":28,"num_read_kb":20337,"num_write":38,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.558462+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:48.558462+0000","last_peered":"2023-06-27T12:39:48.558462+0000","last_clean":"2023-06-27T12:39:48.558462+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:48.558462+0000","last_undegraded":"2023-06-27T12:39:48.558462+0000","last_fullsized":"2023-06-27T12:39:48.558462+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"120'165","reported_seq":377,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.545218+0000","last_change":"2023-06-27T12:39:46.935442+0000","last_active":"2023-06-27T12:39:48.545218+0000","last_peered":"2023-06-27T12:39:48.545218+0000","last_clean":"2023-06-27T12:39:48.545218+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:48.545218+0000","last_undegraded":"2023-06-27T12:39:48.545218+0000","last_fullsized":"2023-06-27T12:39:48.545218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10689252,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":62733,"num_write":120,"num_write_kb":36200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":181,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.554780+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:48.554780+0000","last_peered":"2023-06-27T12:39:48.554780+0000","last_clean":"2023-06-27T12:39:48.554780+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:48.554780+0000","last_undegraded":"2023-06-27T12:39:48.554780+0000","last_fullsized":"2023-06-27T12:39:48.554780+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":308,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.460064+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:48.460064+0000","last_peered":"2023-06-27T12:39:48.460064+0000","last_clean":"2023-06-27T12:39:48.460064+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:48.460064+0000","last_undegraded":"2023-06-27T12:39:48.460064+0000","last_fullsized":"2023-06-27T12:39:48.460064+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"110'112","reported_seq":361,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.556130+0000","last_change":"2023-06-27T12:39:46.457013+0000","last_active":"2023-06-27T12:39:48.556130+0000","last_peered":"2023-06-27T12:39:48.556130+0000","last_clean":"2023-06-27T12:39:48.556130+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:48.556130+0000","last_undegraded":"2023-06-27T12:39:48.556130+0000","last_fullsized":"2023-06-27T12:39:48.556130+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15697672,"num_objects":18,"num_object_clones":12,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":103,"num_read_kb":46395,"num_write":98,"num_write_kb":28357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.442069+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:48.442069+0000","last_peered":"2023-06-27T12:39:48.442069+0000","last_clean":"2023-06-27T12:39:48.442069+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:48.442069+0000","last_undegraded":"2023-06-27T12:39:48.442069+0000","last_fullsized":"2023-06-27T12:39:48.442069+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"118'61","reported_seq":279,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.463584+0000","last_change":"2023-06-27T12:39:46.837150+0000","last_active":"2023-06-27T12:39:48.463584+0000","last_peered":"2023-06-27T12:39:48.463584+0000","last_clean":"2023-06-27T12:39:48.463584+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:48.463584+0000","last_undegraded":"2023-06-27T12:39:48.463584+0000","last_fullsized":"2023-06-27T12:39:48.463584+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3865610,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":29,"num_read_kb":24673,"num_write":45,"num_write_kb":13236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.465356+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:48.465356+0000","last_peered":"2023-06-27T12:39:48.465356+0000","last_clean":"2023-06-27T12:39:48.465356+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:48.465356+0000","last_undegraded":"2023-06-27T12:39:48.465356+0000","last_fullsized":"2023-06-27T12:39:48.465356+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'55","reported_seq":297,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.561934+0000","last_change":"2023-06-27T12:39:46.840035+0000","last_active":"2023-06-27T12:39:48.561934+0000","last_peered":"2023-06-27T12:39:48.561934+0000","last_clean":"2023-06-27T12:39:48.561934+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:48.561934+0000","last_undegraded":"2023-06-27T12:39:48.561934+0000","last_fullsized":"2023-06-27T12:39:48.561934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185042,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":35971,"num_write":52,"num_write_kb":15517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":264,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.464572+0000","last_change":"2023-06-27T12:39:46.836887+0000","last_active":"2023-06-27T12:39:48.464572+0000","last_peered":"2023-06-27T12:39:48.464572+0000","last_clean":"2023-06-27T12:39:48.464572+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:48.464572+0000","last_undegraded":"2023-06-27T12:39:48.464572+0000","last_fullsized":"2023-06-27T12:39:48.464572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":13729,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":185,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.558753+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:48.558753+0000","last_peered":"2023-06-27T12:39:48.558753+0000","last_clean":"2023-06-27T12:39:48.558753+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:48.558753+0000","last_undegraded":"2023-06-27T12:39:48.558753+0000","last_fullsized":"2023-06-27T12:39:48.558753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.557371+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:48.557371+0000","last_peered":"2023-06-27T12:39:48.557371+0000","last_clean":"2023-06-27T12:39:48.557371+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:48.557371+0000","last_undegraded":"2023-06-27T12:39:48.557371+0000","last_fullsized":"2023-06-27T12:39:48.557371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"107'60","reported_seq":292,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-06-27T12:39:48.548896+0000","last_change":"2023-06-27T12:39:46.836198+0000","last_active":"2023-06-27T12:39:48.548896+0000","last_peered":"2023-06-27T12:39:48.548896+0000","last_clean":"2023-06-27T12:39:48.548896+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:48.548896+0000","last_undegraded":"2023-06-27T12:39:48.548896+0000","last_fullsized":"2023-06-27T12:39:48.548896+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9277464,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":37249,"num_write":45,"num_write_kb":13422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":122270760,"num_objects":158,"num_object_clones":104,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":702,"num_read_kb":483860,"num_write":812,"num_write_kb":238241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1026,"ondisk_log_size":1026,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673075,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":164840,"kb_used_data":133920,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94207000,"statfs":{"total":96636764160,"available":96467968000,"internally_reserved":0,"allocated":137134080,"data_stored":156426585,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673075,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":144636,"kb_used_data":113716,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94227204,"statfs":{"total":96636764160,"available":96488656896,"internally_reserved":0,"allocated":116445184,"data_stored":133426819,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705780,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":143412,"kb_used_data":112492,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94228428,"statfs":{"total":96636764160,"available":96489910272,"internally_reserved":0,"allocated":115191808,"data_stored":128380814,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673075,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167440,"kb_used_data":136520,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204400,"statfs":{"total":96636764160,"available":96465305600,"internally_reserved":0,"allocated":139796480,"data_stored":163363075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:50.393 INFO:tasks.rados.rados.0.smithi138.stdout:462: done (2 left) 2023-06-27T12:39:50.393 INFO:tasks.rados.rados.0.smithi138.stdout:463: read oid 9 snap -1 2023-06-27T12:39:50.393 INFO:tasks.rados.rados.0.smithi138.stdout:463: expect (ObjNum 164 snap 25 seq_num 164) 2023-06-27T12:39:50.393 INFO:tasks.rados.rados.0.smithi138.stdout:464: read oid 40 snap -1 2023-06-27T12:39:50.394 INFO:tasks.rados.rados.0.smithi138.stdout:464: expect (ObjNum 122 snap 14 seq_num 122) 2023-06-27T12:39:50.394 INFO:tasks.rados.rados.0.smithi138.stdout:465: write (excl) oid 24 current snap is 58 2023-06-27T12:39:50.394 INFO:tasks.rados.rados.0.smithi138.stdout:465: seq_num 233 ranges {716031=623607,2084775=605732,3298969=617062} 2023-06-27T12:39:50.402 INFO:tasks.rados.rados.0.smithi138.stdout:465: writing smithi138138488-24 from 716031 to 1339638 tid 1 2023-06-27T12:39:50.412 INFO:tasks.rados.rados.0.smithi138.stdout:465: writing smithi138138488-24 from 2084775 to 2690507 tid 2 2023-06-27T12:39:50.422 INFO:tasks.rados.rados.0.smithi138.stdout:465: writing smithi138138488-24 from 3298969 to 3916031 tid 3 2023-06-27T12:39:50.422 INFO:tasks.rados.rados.0.smithi138.stdout:466: read oid 17 snap -1 2023-06-27T12:39:50.422 INFO:tasks.rados.rados.0.smithi138.stdout:466: expect (ObjNum 219 snap 49 seq_num 219) 2023-06-27T12:39:50.423 INFO:tasks.rados.rados.0.smithi138.stdout:467: snap_remove snap 32 2023-06-27T12:39:51.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:51.400 INFO:tasks.rados.rados.0.smithi138.stdout:467: done (6 left) 2023-06-27T12:39:51.400 INFO:tasks.rados.rados.0.smithi138.stdout:468: delete oid 20 current snap is 58 2023-06-27T12:39:51.401 INFO:tasks.rados.rados.0.smithi138.stdout:463: expect (ObjNum 164 snap 25 seq_num 164) 2023-06-27T12:39:51.411 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:51.406+0000 7f3f303bb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:51.414 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:51.410+0000 7f3f303bb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:51.586 INFO:tasks.rados.rados.0.smithi138.stdout:466: expect (ObjNum 219 snap 49 seq_num 219) 2023-06-27T12:39:51.861 INFO:tasks.rados.rados.0.smithi138.stdout:464: expect (ObjNum 122 snap 14 seq_num 122) 2023-06-27T12:39:51.953 INFO:tasks.rados.rados.0.smithi138.stdout:465: finishing write tid 1 to smithi138138488-24 2023-06-27T12:39:51.953 INFO:tasks.rados.rados.0.smithi138.stdout:465: finishing write tid 2 to smithi138138488-24 2023-06-27T12:39:51.954 INFO:tasks.rados.rados.0.smithi138.stdout:465: finishing write tid 3 to smithi138138488-24 2023-06-27T12:39:51.954 INFO:tasks.rados.rados.0.smithi138.stdout:465: finishing write tid 4 to smithi138138488-24 2023-06-27T12:39:51.954 INFO:tasks.rados.rados.0.smithi138.stdout:465: finishing write tid 5 to smithi138138488-24 2023-06-27T12:39:51.954 INFO:tasks.rados.rados.0.smithi138.stdout:465: oid 24 updating version 0 to 115 2023-06-27T12:39:51.954 INFO:tasks.rados.rados.0.smithi138.stdout:465: oid 24 updating version 115 to 119 2023-06-27T12:39:51.954 INFO:tasks.rados.rados.0.smithi138.stdout:465: oid 24 version 119 is already newer than 119 2023-06-27T12:39:51.954 INFO:tasks.rados.rados.0.smithi138.stdout:465: oid 24 version 119 is already newer than 119 2023-06-27T12:39:51.954 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 119 (ObjNum 233 snap 58 seq_num 233) dirty exists 2023-06-27T12:39:51.955 INFO:tasks.rados.rados.0.smithi138.stdout:465: left oid 24 (ObjNum 233 snap 58 seq_num 233) 2023-06-27T12:39:51.955 INFO:tasks.rados.rados.0.smithi138.stdout:460: done (6 left) 2023-06-27T12:39:51.955 INFO:tasks.rados.rados.0.smithi138.stdout:461: done (5 left) 2023-06-27T12:39:51.955 INFO:tasks.rados.rados.0.smithi138.stdout:463: done (4 left) 2023-06-27T12:39:51.955 INFO:tasks.rados.rados.0.smithi138.stdout:464: done (3 left) 2023-06-27T12:39:51.955 INFO:tasks.rados.rados.0.smithi138.stdout:465: done (2 left) 2023-06-27T12:39:51.955 INFO:tasks.rados.rados.0.smithi138.stdout:466: done (1 left) 2023-06-27T12:39:51.955 INFO:tasks.rados.rados.0.smithi138.stdout:468: done (0 left) 2023-06-27T12:39:51.956 INFO:tasks.rados.rados.0.smithi138.stdout:469: read oid 15 snap 55 2023-06-27T12:39:51.956 INFO:tasks.rados.rados.0.smithi138.stdout:469: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:39:51.956 INFO:tasks.rados.rados.0.smithi138.stdout:470: snap_create 2023-06-27T12:39:52.007 INFO:tasks.rados.rados.0.smithi138.stdout:469: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:39:52.457 INFO:tasks.rados.rados.0.smithi138.stdout:469: done (1 left) 2023-06-27T12:39:52.457 INFO:tasks.rados.rados.0.smithi138.stdout:470: done (0 left) 2023-06-27T12:39:52.457 INFO:tasks.rados.rados.0.smithi138.stdout:471: delete oid 12 current snap is 59 2023-06-27T12:39:52.480 INFO:tasks.rados.rados.0.smithi138.stdout:471: done (0 left) 2023-06-27T12:39:52.481 INFO:tasks.rados.rados.0.smithi138.stdout:472: snap_remove snap 52 2023-06-27T12:39:53.457 INFO:tasks.rados.rados.0.smithi138.stdout:472: done (0 left) 2023-06-27T12:39:53.458 INFO:tasks.rados.rados.0.smithi138.stdout:473: snap_create 2023-06-27T12:39:53.505 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:53.637 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:53.632+0000 7f259806c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:53.639 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:53.636+0000 7f259806c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:54.458 INFO:tasks.rados.rados.0.smithi138.stdout:473: done (0 left) 2023-06-27T12:39:54.458 INFO:tasks.rados.rados.0.smithi138.stdout:474: snap_remove snap 47 2023-06-27T12:39:55.334 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:39:55.334 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:39:55.335 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:39:55.465 INFO:tasks.rados.rados.0.smithi138.stdout:474: done (0 left) 2023-06-27T12:39:55.466 INFO:tasks.rados.rados.0.smithi138.stdout:475: read oid 6 snap -1 2023-06-27T12:39:55.466 INFO:tasks.rados.rados.0.smithi138.stdout:475: expect (ObjNum 162 snap 24 seq_num 162) 2023-06-27T12:39:55.466 INFO:tasks.rados.rados.0.smithi138.stdout:476: snap_remove snap 55 2023-06-27T12:39:55.481 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:55.472+0000 7fe3a9755700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:55.485 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:55.481+0000 7fe3a9755700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:55.694 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:39:55.696 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:39:55.708 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2023-06-27T12:39:55.460564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":120508148,"num_objects":162,"num_object_clones":106,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":826,"num_read_kb":507097,"num_write":954,"num_write_kb":241939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1136,"ondisk_log_size":1136,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":607916,"kb_used_data":484236,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376879444,"statfs":{"total":386547056640,"available":385924550656,"internally_reserved":0,"allocated":495857664,"data_stored":569656065,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2352980,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":21,"num_read_kb":23146,"num_write":5,"num_write_kb":1804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.064755"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":228,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.834931+0000","last_change":"2023-06-27T12:39:51.807667+0000","last_active":"2023-06-27T12:39:52.834931+0000","last_peered":"2023-06-27T12:39:52.834931+0000","last_clean":"2023-06-27T12:39:52.834931+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:52.834931+0000","last_undegraded":"2023-06-27T12:39:52.834931+0000","last_fullsized":"2023-06-27T12:39:52.834931+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"123'90","reported_seq":337,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.825864+0000","last_change":"2023-06-27T12:39:51.450995+0000","last_active":"2023-06-27T12:39:52.825864+0000","last_peered":"2023-06-27T12:39:52.825864+0000","last_clean":"2023-06-27T12:39:52.825864+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:52.825864+0000","last_undegraded":"2023-06-27T12:39:52.825864+0000","last_fullsized":"2023-06-27T12:39:52.825864+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6907677,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":51,"num_read_kb":49896,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"118'97","reported_seq":336,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-06-27T12:39:53.466116+0000","last_change":"2023-06-27T12:39:53.496658+0000","last_active":"2023-06-27T12:39:53.466116+0000","last_peered":"2023-06-27T12:39:53.466116+0000","last_clean":"2023-06-27T12:39:53.466116+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:53.466116+0000","last_undegraded":"2023-06-27T12:39:53.466116+0000","last_fullsized":"2023-06-27T12:39:53.466116+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11227076,"num_objects":14,"num_object_clones":9,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":54,"num_read_kb":20449,"num_write":81,"num_write_kb":24054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'40","reported_seq":284,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.833731+0000","last_change":"2023-06-27T12:39:51.837198+0000","last_active":"2023-06-27T12:39:52.833731+0000","last_peered":"2023-06-27T12:39:52.833731+0000","last_clean":"2023-06-27T12:39:52.833731+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:52.833731+0000","last_undegraded":"2023-06-27T12:39:52.833731+0000","last_fullsized":"2023-06-27T12:39:52.833731+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395047,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":52,"num_read_kb":27195,"num_write":35,"num_write_kb":10330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":269,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.810922+0000","last_change":"2023-06-27T12:39:51.994602+0000","last_active":"2023-06-27T12:39:52.810922+0000","last_peered":"2023-06-27T12:39:52.810922+0000","last_clean":"2023-06-27T12:39:52.810922+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:52.810922+0000","last_undegraded":"2023-06-27T12:39:52.810922+0000","last_fullsized":"2023-06-27T12:39:52.810922+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":23,"num_read_kb":23883,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"122'119","reported_seq":364,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.835509+0000","last_change":"2023-06-27T12:39:51.837994+0000","last_active":"2023-06-27T12:39:52.835509+0000","last_peered":"2023-06-27T12:39:52.835509+0000","last_clean":"2023-06-27T12:39:52.835509+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:52.835509+0000","last_undegraded":"2023-06-27T12:39:52.835509+0000","last_fullsized":"2023-06-27T12:39:52.835509+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17462440,"num_objects":19,"num_object_clones":13,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":125,"num_read_kb":87863,"num_write":92,"num_write_kb":26868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":300,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.834654+0000","last_change":"2023-06-27T12:39:51.812161+0000","last_active":"2023-06-27T12:39:52.834654+0000","last_peered":"2023-06-27T12:39:52.834654+0000","last_clean":"2023-06-27T12:39:52.834654+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:52.834654+0000","last_undegraded":"2023-06-27T12:39:52.834654+0000","last_fullsized":"2023-06-27T12:39:52.834654+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":24452,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"117'32","reported_seq":272,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-06-27T12:39:53.464485+0000","last_change":"2023-06-27T12:39:53.494802+0000","last_active":"2023-06-27T12:39:53.464485+0000","last_peered":"2023-06-27T12:39:53.464485+0000","last_clean":"2023-06-27T12:39:53.464485+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:53.464485+0000","last_undegraded":"2023-06-27T12:39:53.464485+0000","last_fullsized":"2023-06-27T12:39:53.464485+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3798382,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25160,"num_write":23,"num_write_kb":7301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":193,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.831094+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:52.831094+0000","last_peered":"2023-06-27T12:39:52.831094+0000","last_clean":"2023-06-27T12:39:52.831094+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:52.831094+0000","last_undegraded":"2023-06-27T12:39:52.831094+0000","last_fullsized":"2023-06-27T12:39:52.831094+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":232,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-06-27T12:39:53.467633+0000","last_change":"2023-06-27T12:39:53.472383+0000","last_active":"2023-06-27T12:39:53.467633+0000","last_peered":"2023-06-27T12:39:53.467633+0000","last_clean":"2023-06-27T12:39:53.467633+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:53.467633+0000","last_undegraded":"2023-06-27T12:39:53.467633+0000","last_fullsized":"2023-06-27T12:39:53.467633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":181,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.838000+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:52.838000+0000","last_peered":"2023-06-27T12:39:52.838000+0000","last_clean":"2023-06-27T12:39:52.838000+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:52.838000+0000","last_undegraded":"2023-06-27T12:39:52.838000+0000","last_fullsized":"2023-06-27T12:39:52.838000+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"100'51","reported_seq":289,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.836359+0000","last_change":"2023-06-27T12:39:51.419302+0000","last_active":"2023-06-27T12:39:52.836359+0000","last_peered":"2023-06-27T12:39:52.836359+0000","last_clean":"2023-06-27T12:39:52.836359+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:52.836359+0000","last_undegraded":"2023-06-27T12:39:52.836359+0000","last_fullsized":"2023-06-27T12:39:52.836359+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168169,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":28,"num_read_kb":20337,"num_write":38,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":189,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-06-27T12:39:53.462766+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:53.462766+0000","last_peered":"2023-06-27T12:39:53.462766+0000","last_clean":"2023-06-27T12:39:53.462766+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:53.462766+0000","last_undegraded":"2023-06-27T12:39:53.462766+0000","last_fullsized":"2023-06-27T12:39:53.462766+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"120'165","reported_seq":387,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.811609+0000","last_change":"2023-06-27T12:39:52.086897+0000","last_active":"2023-06-27T12:39:52.811609+0000","last_peered":"2023-06-27T12:39:52.811609+0000","last_clean":"2023-06-27T12:39:52.811609+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:52.811609+0000","last_undegraded":"2023-06-27T12:39:52.811609+0000","last_fullsized":"2023-06-27T12:39:52.811609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10689252,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":62733,"num_write":120,"num_write_kb":36200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":189,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-06-27T12:39:53.464255+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:53.464255+0000","last_peered":"2023-06-27T12:39:53.464255+0000","last_clean":"2023-06-27T12:39:53.464255+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:53.464255+0000","last_undegraded":"2023-06-27T12:39:53.464255+0000","last_fullsized":"2023-06-27T12:39:53.464255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":316,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.825256+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:52.825256+0000","last_peered":"2023-06-27T12:39:52.825256+0000","last_clean":"2023-06-27T12:39:52.825256+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:52.825256+0000","last_undegraded":"2023-06-27T12:39:52.825256+0000","last_fullsized":"2023-06-27T12:39:52.825256+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"110'112","reported_seq":376,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-06-27T12:39:53.471172+0000","last_change":"2023-06-27T12:39:53.474631+0000","last_active":"2023-06-27T12:39:53.471172+0000","last_peered":"2023-06-27T12:39:53.471172+0000","last_clean":"2023-06-27T12:39:53.471172+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:53.471172+0000","last_undegraded":"2023-06-27T12:39:53.471172+0000","last_fullsized":"2023-06-27T12:39:53.471172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15697672,"num_objects":18,"num_object_clones":12,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":53415,"num_write":98,"num_write_kb":28357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":181,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.834758+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:52.834758+0000","last_peered":"2023-06-27T12:39:52.834758+0000","last_clean":"2023-06-27T12:39:52.834758+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:52.834758+0000","last_undegraded":"2023-06-27T12:39:52.834758+0000","last_fullsized":"2023-06-27T12:39:52.834758+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"118'61","reported_seq":289,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.813871+0000","last_change":"2023-06-27T12:39:52.024811+0000","last_active":"2023-06-27T12:39:52.813871+0000","last_peered":"2023-06-27T12:39:52.813871+0000","last_clean":"2023-06-27T12:39:52.813871+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:52.813871+0000","last_undegraded":"2023-06-27T12:39:52.813871+0000","last_fullsized":"2023-06-27T12:39:52.813871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3865610,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":29,"num_read_kb":24673,"num_write":45,"num_write_kb":13236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":185,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.812696+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:52.812696+0000","last_peered":"2023-06-27T12:39:52.812696+0000","last_clean":"2023-06-27T12:39:52.812696+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:52.812696+0000","last_undegraded":"2023-06-27T12:39:52.812696+0000","last_fullsized":"2023-06-27T12:39:52.812696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'55","reported_seq":307,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.832068+0000","last_change":"2023-06-27T12:39:51.418331+0000","last_active":"2023-06-27T12:39:52.832068+0000","last_peered":"2023-06-27T12:39:52.832068+0000","last_clean":"2023-06-27T12:39:52.832068+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:52.832068+0000","last_undegraded":"2023-06-27T12:39:52.832068+0000","last_fullsized":"2023-06-27T12:39:52.832068+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185042,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":35971,"num_write":52,"num_write_kb":15517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":275,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.806762+0000","last_change":"2023-06-27T12:39:51.972878+0000","last_active":"2023-06-27T12:39:52.806762+0000","last_peered":"2023-06-27T12:39:52.806762+0000","last_clean":"2023-06-27T12:39:52.806762+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:52.806762+0000","last_undegraded":"2023-06-27T12:39:52.806762+0000","last_fullsized":"2023-06-27T12:39:52.806762+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":13729,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":193,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.830675+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:52.830675+0000","last_peered":"2023-06-27T12:39:52.830675+0000","last_clean":"2023-06-27T12:39:52.830675+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:52.830675+0000","last_undegraded":"2023-06-27T12:39:52.830675+0000","last_fullsized":"2023-06-27T12:39:52.830675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":189,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-06-27T12:39:53.467440+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:53.467440+0000","last_peered":"2023-06-27T12:39:53.467440+0000","last_clean":"2023-06-27T12:39:53.467440+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:53.467440+0000","last_undegraded":"2023-06-27T12:39:53.467440+0000","last_fullsized":"2023-06-27T12:39:53.467440+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"107'60","reported_seq":302,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-06-27T12:39:52.832210+0000","last_change":"2023-06-27T12:39:51.459583+0000","last_active":"2023-06-27T12:39:52.832210+0000","last_peered":"2023-06-27T12:39:52.832210+0000","last_clean":"2023-06-27T12:39:52.832210+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:52.832210+0000","last_undegraded":"2023-06-27T12:39:52.832210+0000","last_fullsized":"2023-06-27T12:39:52.832210+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9277464,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":37249,"num_write":45,"num_write_kb":13422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":119917780,"num_objects":160,"num_object_clones":106,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":723,"num_read_kb":507006,"num_write":817,"num_write_kb":240045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1039,"ondisk_log_size":1039,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160304,"kb_used_data":129384,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211536,"statfs":{"total":96636764160,"available":96472612864,"internally_reserved":0,"allocated":132489216,"data_stored":150981197,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":140492,"kb_used_data":109572,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94231348,"statfs":{"total":96636764160,"available":96492900352,"internally_reserved":0,"allocated":112201728,"data_stored":129440680,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":143820,"kb_used_data":112900,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94228020,"statfs":{"total":96636764160,"available":96489492480,"internally_reserved":0,"allocated":115609600,"data_stored":129853787,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163300,"kb_used_data":132380,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94208540,"statfs":{"total":96636764160,"available":96469544960,"internally_reserved":0,"allocated":135557120,"data_stored":159380401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:39:55.711 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:39:55.836 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:55.831+0000 7f3bcd8f8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:55.839 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:55.835+0000 7f3bcd8f8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:56.464 INFO:tasks.rados.rados.0.smithi138.stdout:476: done (1 left) 2023-06-27T12:39:56.465 INFO:tasks.rados.rados.0.smithi138.stdout:477: read oid 25 snap 46 2023-06-27T12:39:56.465 INFO:tasks.rados.rados.0.smithi138.stdout:477: expect (ObjNum 136 snap 17 seq_num 136) 2023-06-27T12:39:56.465 INFO:tasks.rados.rados.0.smithi138.stdout:478: delete oid 2 current snap is 60 2023-06-27T12:39:56.465 INFO:tasks.rados.rados.0.smithi138.stdout:475: expect (ObjNum 162 snap 24 seq_num 162) 2023-06-27T12:39:56.616 INFO:tasks.rados.rados.0.smithi138.stdout:477: expect (ObjNum 136 snap 17 seq_num 136) 2023-06-27T12:39:56.729 INFO:tasks.rados.rados.0.smithi138.stdout:475: done (2 left) 2023-06-27T12:39:56.729 INFO:tasks.rados.rados.0.smithi138.stdout:477: done (1 left) 2023-06-27T12:39:56.729 INFO:tasks.rados.rados.0.smithi138.stdout:478: done (0 left) 2023-06-27T12:39:56.730 INFO:tasks.rados.rados.0.smithi138.stdout:479: read oid 16 snap -1 2023-06-27T12:39:56.730 INFO:tasks.rados.rados.0.smithi138.stdout:479: expect (ObjNum 175 snap 33 seq_num 175) 2023-06-27T12:39:56.730 INFO:tasks.rados.rados.0.smithi138.stdout:480: snap_create 2023-06-27T12:39:56.739 INFO:tasks.rados.rados.0.smithi138.stdout:479: expect (ObjNum 175 snap 33 seq_num 175) 2023-06-27T12:39:57.484 INFO:tasks.rados.rados.0.smithi138.stdout:479: done (1 left) 2023-06-27T12:39:57.484 INFO:tasks.rados.rados.0.smithi138.stdout:480: done (0 left) 2023-06-27T12:39:57.484 INFO:tasks.rados.rados.0.smithi138.stdout:481: snap_create 2023-06-27T12:39:57.961 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:39:58.095 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:58.090+0000 7f65c895d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:58.097 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:39:58.094+0000 7f65c895d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:39:58.490 INFO:tasks.rados.rados.0.smithi138.stdout:481: done (0 left) 2023-06-27T12:39:58.490 INFO:tasks.rados.rados.0.smithi138.stdout:482: snap_remove snap 61 2023-06-27T12:39:59.490 INFO:tasks.rados.rados.0.smithi138.stdout:482: done (0 left) 2023-06-27T12:39:59.491 INFO:tasks.rados.rados.0.smithi138.stdout:483: delete oid 48 current snap is 62 2023-06-27T12:39:59.580 INFO:tasks.rados.rados.0.smithi138.stdout:483: done (0 left) 2023-06-27T12:39:59.580 INFO:tasks.rados.rados.0.smithi138.stdout:484: read oid 46 snap -1 2023-06-27T12:39:59.580 INFO:tasks.rados.rados.0.smithi138.stdout:484: expect deleted 2023-06-27T12:39:59.580 INFO:tasks.rados.rados.0.smithi138.stdout:485: write oid 40 current snap is 62 2023-06-27T12:39:59.581 INFO:tasks.rados.rados.0.smithi138.stdout:485: seq_num 238 ranges {400066=716205,1860832=139234} 2023-06-27T12:39:59.587 INFO:tasks.rados.rados.0.smithi138.stdout:485: writing smithi138138488-40 from 400066 to 1116271 tid 1 2023-06-27T12:39:59.593 INFO:tasks.rados.rados.0.smithi138.stdout:485: writing smithi138138488-40 from 1860832 to 2000066 tid 2 2023-06-27T12:39:59.593 INFO:tasks.rados.rados.0.smithi138.stdout:486: snap_create 2023-06-27T12:39:59.670 INFO:tasks.rados.rados.0.smithi138.stdout:485: finishing write tid 1 to smithi138138488-40 2023-06-27T12:39:59.670 INFO:tasks.rados.rados.0.smithi138.stdout:485: finishing write tid 2 to smithi138138488-40 2023-06-27T12:39:59.671 INFO:tasks.rados.rados.0.smithi138.stdout:485: finishing write tid 3 to smithi138138488-40 2023-06-27T12:39:59.672 INFO:tasks.rados.rados.0.smithi138.stdout:485: finishing write tid 4 to smithi138138488-40 2023-06-27T12:39:59.673 INFO:tasks.rados.rados.0.smithi138.stdout:485: oid 40 updating version 0 to 47 2023-06-27T12:39:59.673 INFO:tasks.rados.rados.0.smithi138.stdout:485: oid 40 version 47 is already newer than 47 2023-06-27T12:39:59.673 INFO:tasks.rados.rados.0.smithi138.stdout:485: oid 40 version 47 is already newer than 47 2023-06-27T12:39:59.673 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 47 (ObjNum 238 snap 62 seq_num 238) dirty exists 2023-06-27T12:39:59.673 INFO:tasks.rados.rados.0.smithi138.stdout:485: left oid 40 (ObjNum 238 snap 62 seq_num 238) 2023-06-27T12:40:00.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:40:00.308 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:00.303+0000 7f4236d20700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:00.312 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:00.308+0000 7f4236d20700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:00.395 INFO:tasks.rados.rados.0.smithi138.stdout:484: done (2 left) 2023-06-27T12:40:00.396 INFO:tasks.rados.rados.0.smithi138.stdout:485: done (1 left) 2023-06-27T12:40:00.396 INFO:tasks.rados.rados.0.smithi138.stdout:486: done (0 left) 2023-06-27T12:40:00.396 INFO:tasks.rados.rados.0.smithi138.stdout:487: snap_create 2023-06-27T12:40:00.711 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:00.711 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:00.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:00.841 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:00.834+0000 7f59a5106700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:00.844 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:00.840+0000 7f59a5106700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:01.038 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:01.040 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:01.051 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2023-06-27T12:40:00.391226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":120508148,"num_objects":162,"num_object_clones":106,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":837,"num_read_kb":520279,"num_write":954,"num_write_kb":241939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1153,"ondisk_log_size":1153,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":608008,"kb_used_data":484328,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376879352,"statfs":{"total":386547056640,"available":385924456448,"internally_reserved":0,"allocated":495951872,"data_stored":569735616,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":13182,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.931432"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":239,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.830451+0000","last_change":"2023-06-27T12:39:56.556026+0000","last_active":"2023-06-27T12:39:58.830451+0000","last_peered":"2023-06-27T12:39:58.830451+0000","last_clean":"2023-06-27T12:39:58.830451+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:39:58.830451+0000","last_undegraded":"2023-06-27T12:39:58.830451+0000","last_fullsized":"2023-06-27T12:39:58.830451+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"123'90","reported_seq":349,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.837576+0000","last_change":"2023-06-27T12:39:56.639827+0000","last_active":"2023-06-27T12:39:58.837576+0000","last_peered":"2023-06-27T12:39:58.837576+0000","last_clean":"2023-06-27T12:39:58.837576+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:39:58.837576+0000","last_undegraded":"2023-06-27T12:39:58.837576+0000","last_fullsized":"2023-06-27T12:39:58.837576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6907677,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":51,"num_read_kb":49896,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"118'97","reported_seq":347,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.491530+0000","last_change":"2023-06-27T12:39:57.808661+0000","last_active":"2023-06-27T12:39:58.491530+0000","last_peered":"2023-06-27T12:39:58.491530+0000","last_clean":"2023-06-27T12:39:58.491530+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:39:58.491530+0000","last_undegraded":"2023-06-27T12:39:58.491530+0000","last_fullsized":"2023-06-27T12:39:58.491530+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11227076,"num_objects":14,"num_object_clones":9,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":54,"num_read_kb":20449,"num_write":81,"num_write_kb":24054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'40","reported_seq":295,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.829461+0000","last_change":"2023-06-27T12:39:56.571605+0000","last_active":"2023-06-27T12:39:58.829461+0000","last_peered":"2023-06-27T12:39:58.829461+0000","last_clean":"2023-06-27T12:39:58.829461+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:39:58.829461+0000","last_undegraded":"2023-06-27T12:39:58.829461+0000","last_fullsized":"2023-06-27T12:39:58.829461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395047,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":52,"num_read_kb":27195,"num_write":35,"num_write_kb":10330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":280,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.806593+0000","last_change":"2023-06-27T12:39:56.481435+0000","last_active":"2023-06-27T12:39:58.806593+0000","last_peered":"2023-06-27T12:39:58.806593+0000","last_clean":"2023-06-27T12:39:58.806593+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:39:58.806593+0000","last_undegraded":"2023-06-27T12:39:58.806593+0000","last_fullsized":"2023-06-27T12:39:58.806593+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":23,"num_read_kb":23883,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"122'119","reported_seq":379,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.829899+0000","last_change":"2023-06-27T12:39:57.500312+0000","last_active":"2023-06-27T12:39:58.829899+0000","last_peered":"2023-06-27T12:39:58.829899+0000","last_clean":"2023-06-27T12:39:58.829899+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:39:58.829899+0000","last_undegraded":"2023-06-27T12:39:58.829899+0000","last_fullsized":"2023-06-27T12:39:58.829899+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17462440,"num_objects":19,"num_object_clones":13,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":132,"num_read_kb":92872,"num_write":92,"num_write_kb":26868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":311,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.830144+0000","last_change":"2023-06-27T12:39:56.570142+0000","last_active":"2023-06-27T12:39:58.830144+0000","last_peered":"2023-06-27T12:39:58.830144+0000","last_clean":"2023-06-27T12:39:58.830144+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:39:58.830144+0000","last_undegraded":"2023-06-27T12:39:58.830144+0000","last_fullsized":"2023-06-27T12:39:58.830144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":24452,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"117'32","reported_seq":282,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.501713+0000","last_change":"2023-06-27T12:39:57.808076+0000","last_active":"2023-06-27T12:39:58.501713+0000","last_peered":"2023-06-27T12:39:58.501713+0000","last_clean":"2023-06-27T12:39:58.501713+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:39:58.501713+0000","last_undegraded":"2023-06-27T12:39:58.501713+0000","last_fullsized":"2023-06-27T12:39:58.501713+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3798382,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25160,"num_write":23,"num_write_kb":7301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":201,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.829921+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:39:58.829921+0000","last_peered":"2023-06-27T12:39:58.829921+0000","last_clean":"2023-06-27T12:39:58.829921+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:39:58.829921+0000","last_undegraded":"2023-06-27T12:39:58.829921+0000","last_fullsized":"2023-06-27T12:39:58.829921+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":242,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.498959+0000","last_change":"2023-06-27T12:39:56.490241+0000","last_active":"2023-06-27T12:39:58.498959+0000","last_peered":"2023-06-27T12:39:58.498959+0000","last_clean":"2023-06-27T12:39:58.498959+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:39:58.498959+0000","last_undegraded":"2023-06-27T12:39:58.498959+0000","last_fullsized":"2023-06-27T12:39:58.498959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":189,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.831286+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:39:58.831286+0000","last_peered":"2023-06-27T12:39:58.831286+0000","last_clean":"2023-06-27T12:39:58.831286+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:39:58.831286+0000","last_undegraded":"2023-06-27T12:39:58.831286+0000","last_fullsized":"2023-06-27T12:39:58.831286+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"100'51","reported_seq":302,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.841158+0000","last_change":"2023-06-27T12:39:56.585246+0000","last_active":"2023-06-27T12:39:58.841158+0000","last_peered":"2023-06-27T12:39:58.841158+0000","last_clean":"2023-06-27T12:39:58.841158+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:39:58.841158+0000","last_undegraded":"2023-06-27T12:39:58.841158+0000","last_fullsized":"2023-06-27T12:39:58.841158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168169,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":24591,"num_write":38,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":197,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.497279+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:39:58.497279+0000","last_peered":"2023-06-27T12:39:58.497279+0000","last_clean":"2023-06-27T12:39:58.497279+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:39:58.497279+0000","last_undegraded":"2023-06-27T12:39:58.497279+0000","last_fullsized":"2023-06-27T12:39:58.497279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"128'182","reported_seq":398,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.811361+0000","last_change":"2023-06-27T12:39:57.820603+0000","last_active":"2023-06-27T12:39:58.811361+0000","last_peered":"2023-06-27T12:39:58.811361+0000","last_clean":"2023-06-27T12:39:58.811361+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:39:58.811361+0000","last_undegraded":"2023-06-27T12:39:58.811361+0000","last_fullsized":"2023-06-27T12:39:58.811361+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10689252,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":62733,"num_write":120,"num_write_kb":36200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":197,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.497528+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:39:58.497528+0000","last_peered":"2023-06-27T12:39:58.497528+0000","last_clean":"2023-06-27T12:39:58.497528+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:39:58.497528+0000","last_undegraded":"2023-06-27T12:39:58.497528+0000","last_fullsized":"2023-06-27T12:39:58.497528+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":324,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.834958+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:39:58.834958+0000","last_peered":"2023-06-27T12:39:58.834958+0000","last_clean":"2023-06-27T12:39:58.834958+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:39:58.834958+0000","last_undegraded":"2023-06-27T12:39:58.834958+0000","last_fullsized":"2023-06-27T12:39:58.834958+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"110'112","reported_seq":387,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.498536+0000","last_change":"2023-06-27T12:39:57.806935+0000","last_active":"2023-06-27T12:39:58.498536+0000","last_peered":"2023-06-27T12:39:58.498536+0000","last_clean":"2023-06-27T12:39:58.498536+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:39:58.498536+0000","last_undegraded":"2023-06-27T12:39:58.498536+0000","last_fullsized":"2023-06-27T12:39:58.498536+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15697672,"num_objects":18,"num_object_clones":12,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":53415,"num_write":98,"num_write_kb":28357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":189,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.833327+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:39:58.833327+0000","last_peered":"2023-06-27T12:39:58.833327+0000","last_clean":"2023-06-27T12:39:58.833327+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:39:58.833327+0000","last_undegraded":"2023-06-27T12:39:58.833327+0000","last_fullsized":"2023-06-27T12:39:58.833327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"118'61","reported_seq":300,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.813160+0000","last_change":"2023-06-27T12:39:56.480136+0000","last_active":"2023-06-27T12:39:58.813160+0000","last_peered":"2023-06-27T12:39:58.813160+0000","last_clean":"2023-06-27T12:39:58.813160+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:39:58.813160+0000","last_undegraded":"2023-06-27T12:39:58.813160+0000","last_fullsized":"2023-06-27T12:39:58.813160+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3865610,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":29,"num_read_kb":24673,"num_write":45,"num_write_kb":13236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":193,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.814045+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:39:58.814045+0000","last_peered":"2023-06-27T12:39:58.814045+0000","last_clean":"2023-06-27T12:39:58.814045+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:39:58.814045+0000","last_undegraded":"2023-06-27T12:39:58.814045+0000","last_fullsized":"2023-06-27T12:39:58.814045+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'55","reported_seq":318,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.836931+0000","last_change":"2023-06-27T12:39:56.581341+0000","last_active":"2023-06-27T12:39:58.836931+0000","last_peered":"2023-06-27T12:39:58.836931+0000","last_clean":"2023-06-27T12:39:58.836931+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:39:58.836931+0000","last_undegraded":"2023-06-27T12:39:58.836931+0000","last_fullsized":"2023-06-27T12:39:58.836931+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185042,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":35971,"num_write":52,"num_write_kb":15517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":288,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.808588+0000","last_change":"2023-06-27T12:39:56.480984+0000","last_active":"2023-06-27T12:39:58.808588+0000","last_peered":"2023-06-27T12:39:58.808588+0000","last_clean":"2023-06-27T12:39:58.808588+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:39:58.808588+0000","last_undegraded":"2023-06-27T12:39:58.808588+0000","last_fullsized":"2023-06-27T12:39:58.808588+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":201,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.829584+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:39:58.829584+0000","last_peered":"2023-06-27T12:39:58.829584+0000","last_clean":"2023-06-27T12:39:58.829584+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:39:58.829584+0000","last_undegraded":"2023-06-27T12:39:58.829584+0000","last_fullsized":"2023-06-27T12:39:58.829584+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.502270+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:39:58.502270+0000","last_peered":"2023-06-27T12:39:58.502270+0000","last_clean":"2023-06-27T12:39:58.502270+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:39:58.502270+0000","last_undegraded":"2023-06-27T12:39:58.502270+0000","last_fullsized":"2023-06-27T12:39:58.502270+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"107'60","reported_seq":313,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-06-27T12:39:58.832834+0000","last_change":"2023-06-27T12:39:56.581503+0000","last_active":"2023-06-27T12:39:58.832834+0000","last_peered":"2023-06-27T12:39:58.832834+0000","last_clean":"2023-06-27T12:39:58.832834+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:39:58.832834+0000","last_undegraded":"2023-06-27T12:39:58.832834+0000","last_fullsized":"2023-06-27T12:39:58.832834+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9277464,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":37249,"num_write":45,"num_write_kb":13422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":119917780,"num_objects":160,"num_object_clones":106,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":734,"num_read_kb":520188,"num_write":817,"num_write_kb":240045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1056,"ondisk_log_size":1056,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160328,"kb_used_data":129408,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211512,"statfs":{"total":96636764160,"available":96472588288,"internally_reserved":0,"allocated":132513792,"data_stored":151001951,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":140516,"kb_used_data":109596,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94231324,"statfs":{"total":96636764160,"available":96492875776,"internally_reserved":0,"allocated":112226304,"data_stored":129461434,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705790,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":143844,"kb_used_data":112924,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94227996,"statfs":{"total":96636764160,"available":96489467904,"internally_reserved":0,"allocated":115634176,"data_stored":129874541,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163320,"kb_used_data":132400,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94208520,"statfs":{"total":96636764160,"available":96469524480,"internally_reserved":0,"allocated":135577600,"data_stored":159397690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:01.396 INFO:tasks.rados.rados.0.smithi138.stdout:487: done (0 left) 2023-06-27T12:40:01.397 INFO:tasks.rados.rados.0.smithi138.stdout:488: write (excl) oid 9 current snap is 64 2023-06-27T12:40:01.397 INFO:tasks.rados.rados.0.smithi138.stdout:488: seq_num 239 ranges {416873=791454,1919314=97559} 2023-06-27T12:40:01.406 INFO:tasks.rados.rados.0.smithi138.stdout:488: writing smithi138138488-9 from 416873 to 1208327 tid 1 2023-06-27T12:40:01.411 INFO:tasks.rados.rados.0.smithi138.stdout:488: writing smithi138138488-9 from 1919314 to 2016873 tid 2 2023-06-27T12:40:01.411 INFO:tasks.rados.rados.0.smithi138.stdout:489: write oid 41 current snap is 64 2023-06-27T12:40:01.411 INFO:tasks.rados.rados.0.smithi138.stdout:489: seq_num 240 ranges {433680=466703,1577796=455884} 2023-06-27T12:40:01.418 INFO:tasks.rados.rados.0.smithi138.stdout:489: writing smithi138138488-41 from 433680 to 900383 tid 1 2023-06-27T12:40:01.424 INFO:tasks.rados.rados.0.smithi138.stdout:489: writing smithi138138488-41 from 1577796 to 2033680 tid 2 2023-06-27T12:40:01.425 INFO:tasks.rados.rados.0.smithi138.stdout:490: snap_create 2023-06-27T12:40:01.448 INFO:tasks.rados.rados.0.smithi138.stdout:488: finishing write tid 1 to smithi138138488-9 2023-06-27T12:40:01.462 INFO:tasks.rados.rados.0.smithi138.stdout:488: finishing write tid 2 to smithi138138488-9 2023-06-27T12:40:01.469 INFO:tasks.rados.rados.0.smithi138.stdout:488: finishing write tid 3 to smithi138138488-9 2023-06-27T12:40:01.470 INFO:tasks.rados.rados.0.smithi138.stdout:488: finishing write tid 4 to smithi138138488-9 2023-06-27T12:40:01.470 INFO:tasks.rados.rados.0.smithi138.stdout:488: oid 9 updating version 0 to 115 2023-06-27T12:40:01.470 INFO:tasks.rados.rados.0.smithi138.stdout:488: oid 9 updating version 115 to 118 2023-06-27T12:40:01.471 INFO:tasks.rados.rados.0.smithi138.stdout:488: oid 9 version 118 is already newer than 118 2023-06-27T12:40:01.471 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 118 (ObjNum 239 snap 64 seq_num 239) dirty exists 2023-06-27T12:40:01.471 INFO:tasks.rados.rados.0.smithi138.stdout:488: left oid 9 (ObjNum 239 snap 64 seq_num 239) 2023-06-27T12:40:01.518 INFO:tasks.rados.rados.0.smithi138.stdout:489: finishing write tid 1 to smithi138138488-41 2023-06-27T12:40:01.520 INFO:tasks.rados.rados.0.smithi138.stdout:489: finishing write tid 2 to smithi138138488-41 2023-06-27T12:40:01.521 INFO:tasks.rados.rados.0.smithi138.stdout:489: finishing write tid 3 to smithi138138488-41 2023-06-27T12:40:01.522 INFO:tasks.rados.rados.0.smithi138.stdout:489: finishing write tid 4 to smithi138138488-41 2023-06-27T12:40:01.523 INFO:tasks.rados.rados.0.smithi138.stdout:489: oid 41 updating version 0 to 59 2023-06-27T12:40:01.523 INFO:tasks.rados.rados.0.smithi138.stdout:489: oid 41 version 59 is already newer than 59 2023-06-27T12:40:01.523 INFO:tasks.rados.rados.0.smithi138.stdout:489: oid 41 version 59 is already newer than 59 2023-06-27T12:40:01.523 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 41 v 59 (ObjNum 240 snap 64 seq_num 240) dirty exists 2023-06-27T12:40:01.523 INFO:tasks.rados.rados.0.smithi138.stdout:489: left oid 41 (ObjNum 240 snap 64 seq_num 240) 2023-06-27T12:40:02.384 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:40:02.396 INFO:tasks.rados.rados.0.smithi138.stdout:488: done (2 left) 2023-06-27T12:40:02.396 INFO:tasks.rados.rados.0.smithi138.stdout:489: done (1 left) 2023-06-27T12:40:02.397 INFO:tasks.rados.rados.0.smithi138.stdout:490: done (0 left) 2023-06-27T12:40:02.397 INFO:tasks.rados.rados.0.smithi138.stdout:491: snap_remove snap 59 2023-06-27T12:40:02.518 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:02.512+0000 7f7f0c8d1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:02.521 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:02.517+0000 7f7f0c8d1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:03.396 INFO:tasks.rados.rados.0.smithi138.stdout:491: done (0 left) 2023-06-27T12:40:03.396 INFO:tasks.rados.rados.0.smithi138.stdout:492: snap_create 2023-06-27T12:40:04.397 INFO:tasks.rados.rados.0.smithi138.stdout:492: done (0 left) 2023-06-27T12:40:04.398 INFO:tasks.rados.rados.0.smithi138.stdout:493: read oid 32 snap -1 2023-06-27T12:40:04.398 INFO:tasks.rados.rados.0.smithi138.stdout:493: expect (ObjNum 173 snap 29 seq_num 173) 2023-06-27T12:40:04.398 INFO:tasks.rados.rados.0.smithi138.stdout:494: write oid 40 current snap is 66 2023-06-27T12:40:04.398 INFO:tasks.rados.rados.0.smithi138.stdout:494: seq_num 241 ranges {450487=541952,1636278=414209} 2023-06-27T12:40:04.403 INFO:tasks.rados.rados.0.smithi138.stdout:494: writing smithi138138488-40 from 450487 to 992439 tid 1 2023-06-27T12:40:04.410 INFO:tasks.rados.rados.0.smithi138.stdout:494: writing smithi138138488-40 from 1636278 to 2050487 tid 2 2023-06-27T12:40:04.410 INFO:tasks.rados.rados.0.smithi138.stdout:495: snap_create 2023-06-27T12:40:04.442 INFO:tasks.rados.rados.0.smithi138.stdout:493: expect (ObjNum 173 snap 29 seq_num 173) 2023-06-27T12:40:04.560 INFO:tasks.rados.rados.0.smithi138.stdout:494: finishing write tid 1 to smithi138138488-40 2023-06-27T12:40:04.560 INFO:tasks.rados.rados.0.smithi138.stdout:494: finishing write tid 2 to smithi138138488-40 2023-06-27T12:40:04.561 INFO:tasks.rados.rados.0.smithi138.stdout:494: finishing write tid 3 to smithi138138488-40 2023-06-27T12:40:04.561 INFO:tasks.rados.rados.0.smithi138.stdout:494: finishing write tid 4 to smithi138138488-40 2023-06-27T12:40:04.561 INFO:tasks.rados.rados.0.smithi138.stdout:494: oid 40 updating version 0 to 51 2023-06-27T12:40:04.561 INFO:tasks.rados.rados.0.smithi138.stdout:494: oid 40 version 51 is already newer than 51 2023-06-27T12:40:04.561 INFO:tasks.rados.rados.0.smithi138.stdout:494: oid 40 version 51 is already newer than 51 2023-06-27T12:40:04.561 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 51 (ObjNum 241 snap 66 seq_num 241) dirty exists 2023-06-27T12:40:04.561 INFO:tasks.rados.rados.0.smithi138.stdout:494: left oid 40 (ObjNum 241 snap 66 seq_num 241) 2023-06-27T12:40:04.593 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:40:04.720 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:04.715+0000 7f6000a8a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:04.723 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:04.720+0000 7f6000a8a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:05.397 INFO:tasks.rados.rados.0.smithi138.stdout:493: done (2 left) 2023-06-27T12:40:05.398 INFO:tasks.rados.rados.0.smithi138.stdout:494: done (1 left) 2023-06-27T12:40:05.398 INFO:tasks.rados.rados.0.smithi138.stdout:495: done (0 left) 2023-06-27T12:40:05.398 INFO:tasks.rados.rados.0.smithi138.stdout:496: read oid 38 snap -1 2023-06-27T12:40:05.398 INFO:tasks.rados.rados.0.smithi138.stdout:496: expect deleted 2023-06-27T12:40:05.398 INFO:tasks.rados.rados.0.smithi138.stdout:497: snap_remove snap 56 2023-06-27T12:40:06.054 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:06.054 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:06.055 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:06.190 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:06.183+0000 7f7dd178a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:06.193 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:06.189+0000 7f7dd178a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:06.385 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:06.387 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:06.398 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2023-06-27T12:40:05.930620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":114222462,"num_objects":166,"num_object_clones":110,"num_object_copies":498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":842,"num_read_kb":520282,"num_write":966,"num_write_kb":244546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1180,"ondisk_log_size":1180,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":615972,"kb_used_data":492292,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376871388,"statfs":{"total":386547056640,"available":385916301312,"internally_reserved":0,"allocated":504107008,"data_stored":582749323,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6285686,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":2607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000853"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":249,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.848154+0000","last_change":"2023-06-27T12:40:03.852985+0000","last_active":"2023-06-27T12:40:03.848154+0000","last_peered":"2023-06-27T12:40:03.848154+0000","last_clean":"2023-06-27T12:40:03.848154+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:03.848154+0000","last_undegraded":"2023-06-27T12:40:03.848154+0000","last_fullsized":"2023-06-27T12:40:03.848154+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"123'90","reported_seq":357,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.891375+0000","last_change":"2023-06-27T12:40:03.911696+0000","last_active":"2023-06-27T12:40:03.891375+0000","last_peered":"2023-06-27T12:40:03.891375+0000","last_clean":"2023-06-27T12:40:03.891375+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:03.891375+0000","last_undegraded":"2023-06-27T12:40:03.891375+0000","last_fullsized":"2023-06-27T12:40:03.891375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6907677,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":51,"num_read_kb":49896,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"118'97","reported_seq":355,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.408807+0000","last_change":"2023-06-27T12:40:03.415227+0000","last_active":"2023-06-27T12:40:03.408807+0000","last_peered":"2023-06-27T12:40:03.408807+0000","last_clean":"2023-06-27T12:40:03.408807+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:03.408807+0000","last_undegraded":"2023-06-27T12:40:03.408807+0000","last_fullsized":"2023-06-27T12:40:03.408807+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11227076,"num_objects":14,"num_object_clones":9,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":54,"num_read_kb":20449,"num_write":81,"num_write_kb":24054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"131'47","reported_seq":310,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.856142+0000","last_change":"2023-06-27T12:40:03.856275+0000","last_active":"2023-06-27T12:40:03.856142+0000","last_peered":"2023-06-27T12:40:03.856142+0000","last_clean":"2023-06-27T12:40:03.856142+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:03.856142+0000","last_undegraded":"2023-06-27T12:40:03.856142+0000","last_fullsized":"2023-06-27T12:40:03.856142+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5344659,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":27196,"num_write":39,"num_write_kb":11166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":288,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.913272+0000","last_change":"2023-06-27T12:40:03.922612+0000","last_active":"2023-06-27T12:40:03.913272+0000","last_peered":"2023-06-27T12:40:03.913272+0000","last_clean":"2023-06-27T12:40:03.913272+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:03.913272+0000","last_undegraded":"2023-06-27T12:40:03.913272+0000","last_fullsized":"2023-06-27T12:40:03.913272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":23,"num_read_kb":23883,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"122'119","reported_seq":389,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.851124+0000","last_change":"2023-06-27T12:40:03.854949+0000","last_active":"2023-06-27T12:40:03.851124+0000","last_peered":"2023-06-27T12:40:03.851124+0000","last_clean":"2023-06-27T12:40:03.851124+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:03.851124+0000","last_undegraded":"2023-06-27T12:40:03.851124+0000","last_fullsized":"2023-06-27T12:40:03.851124+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17462440,"num_objects":19,"num_object_clones":13,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":132,"num_read_kb":92872,"num_write":92,"num_write_kb":26868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":321,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.843263+0000","last_change":"2023-06-27T12:40:03.854142+0000","last_active":"2023-06-27T12:40:03.843263+0000","last_peered":"2023-06-27T12:40:03.843263+0000","last_clean":"2023-06-27T12:40:03.843263+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:03.843263+0000","last_undegraded":"2023-06-27T12:40:03.843263+0000","last_fullsized":"2023-06-27T12:40:03.843263+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":24452,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"117'32","reported_seq":290,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.409059+0000","last_change":"2023-06-27T12:40:03.415523+0000","last_active":"2023-06-27T12:40:03.409059+0000","last_peered":"2023-06-27T12:40:03.409059+0000","last_clean":"2023-06-27T12:40:03.409059+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:03.409059+0000","last_undegraded":"2023-06-27T12:40:03.409059+0000","last_fullsized":"2023-06-27T12:40:03.409059+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3798382,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25160,"num_write":23,"num_write_kb":7301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":207,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.882913+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:03.882913+0000","last_peered":"2023-06-27T12:40:03.882913+0000","last_clean":"2023-06-27T12:40:03.882913+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:03.882913+0000","last_undegraded":"2023-06-27T12:40:03.882913+0000","last_fullsized":"2023-06-27T12:40:03.882913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":250,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.409606+0000","last_change":"2023-06-27T12:40:03.415720+0000","last_active":"2023-06-27T12:40:03.409606+0000","last_peered":"2023-06-27T12:40:03.409606+0000","last_clean":"2023-06-27T12:40:03.409606+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:03.409606+0000","last_undegraded":"2023-06-27T12:40:03.409606+0000","last_fullsized":"2023-06-27T12:40:03.409606+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":197,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.850982+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:03.850982+0000","last_peered":"2023-06-27T12:40:03.850982+0000","last_clean":"2023-06-27T12:40:03.850982+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:03.850982+0000","last_undegraded":"2023-06-27T12:40:03.850982+0000","last_fullsized":"2023-06-27T12:40:03.850982+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"100'51","reported_seq":310,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.901655+0000","last_change":"2023-06-27T12:40:03.910792+0000","last_active":"2023-06-27T12:40:03.901655+0000","last_peered":"2023-06-27T12:40:03.901655+0000","last_clean":"2023-06-27T12:40:03.901655+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:03.901655+0000","last_undegraded":"2023-06-27T12:40:03.901655+0000","last_fullsized":"2023-06-27T12:40:03.901655+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168169,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":24591,"num_write":38,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.412342+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:03.412342+0000","last_peered":"2023-06-27T12:40:03.412342+0000","last_clean":"2023-06-27T12:40:03.412342+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:03.412342+0000","last_undegraded":"2023-06-27T12:40:03.412342+0000","last_fullsized":"2023-06-27T12:40:03.412342+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"128'182","reported_seq":406,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.899961+0000","last_change":"2023-06-27T12:40:03.914224+0000","last_active":"2023-06-27T12:40:03.899961+0000","last_peered":"2023-06-27T12:40:03.899961+0000","last_clean":"2023-06-27T12:40:03.899961+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:03.899961+0000","last_undegraded":"2023-06-27T12:40:03.899961+0000","last_fullsized":"2023-06-27T12:40:03.899961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10689252,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":62733,"num_write":120,"num_write_kb":36200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":203,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.409146+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:03.409146+0000","last_peered":"2023-06-27T12:40:03.409146+0000","last_clean":"2023-06-27T12:40:03.409146+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:03.409146+0000","last_undegraded":"2023-06-27T12:40:03.409146+0000","last_fullsized":"2023-06-27T12:40:03.409146+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":330,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.890781+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:03.890781+0000","last_peered":"2023-06-27T12:40:03.890781+0000","last_clean":"2023-06-27T12:40:03.890781+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:03.890781+0000","last_undegraded":"2023-06-27T12:40:03.890781+0000","last_fullsized":"2023-06-27T12:40:03.890781+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"133'118","reported_seq":399,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.408393+0000","last_change":"2023-06-27T12:40:03.462106+0000","last_active":"2023-06-27T12:40:03.408393+0000","last_peered":"2023-06-27T12:40:03.408393+0000","last_clean":"2023-06-27T12:40:03.408393+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:03.408393+0000","last_undegraded":"2023-06-27T12:40:03.408393+0000","last_fullsized":"2023-06-27T12:40:03.408393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958197,"num_objects":19,"num_object_clones":13,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":112,"num_read_kb":53416,"num_write":102,"num_write_kb":29226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":197,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.852499+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:03.852499+0000","last_peered":"2023-06-27T12:40:03.852499+0000","last_clean":"2023-06-27T12:40:03.852499+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:03.852499+0000","last_undegraded":"2023-06-27T12:40:03.852499+0000","last_fullsized":"2023-06-27T12:40:03.852499+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"118'61","reported_seq":308,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.905737+0000","last_change":"2023-06-27T12:40:03.911786+0000","last_active":"2023-06-27T12:40:03.905737+0000","last_peered":"2023-06-27T12:40:03.905737+0000","last_clean":"2023-06-27T12:40:03.905737+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:03.905737+0000","last_undegraded":"2023-06-27T12:40:03.905737+0000","last_fullsized":"2023-06-27T12:40:03.905737+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3865610,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":29,"num_read_kb":24673,"num_write":45,"num_write_kb":13236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":199,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.906453+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:03.906453+0000","last_peered":"2023-06-27T12:40:03.906453+0000","last_clean":"2023-06-27T12:40:03.906453+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:03.906453+0000","last_undegraded":"2023-06-27T12:40:03.906453+0000","last_fullsized":"2023-06-27T12:40:03.906453+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'59","reported_seq":330,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.889859+0000","last_change":"2023-06-27T12:40:03.964843+0000","last_active":"2023-06-27T12:40:03.889859+0000","last_peered":"2023-06-27T12:40:03.889859+0000","last_clean":"2023-06-27T12:40:03.889859+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:03.889859+0000","last_undegraded":"2023-06-27T12:40:03.889859+0000","last_fullsized":"2023-06-27T12:40:03.889859+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12319531,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":35972,"num_write":56,"num_write_kb":16419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":296,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.908355+0000","last_change":"2023-06-27T12:40:03.912457+0000","last_active":"2023-06-27T12:40:03.908355+0000","last_peered":"2023-06-27T12:40:03.908355+0000","last_clean":"2023-06-27T12:40:03.908355+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:03.908355+0000","last_undegraded":"2023-06-27T12:40:03.908355+0000","last_fullsized":"2023-06-27T12:40:03.908355+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":207,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.895859+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:03.895859+0000","last_peered":"2023-06-27T12:40:03.895859+0000","last_clean":"2023-06-27T12:40:03.895859+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:03.895859+0000","last_undegraded":"2023-06-27T12:40:03.895859+0000","last_fullsized":"2023-06-27T12:40:03.895859+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":203,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.409769+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:03.409769+0000","last_peered":"2023-06-27T12:40:03.409769+0000","last_clean":"2023-06-27T12:40:03.409769+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:03.409769+0000","last_undegraded":"2023-06-27T12:40:03.409769+0000","last_fullsized":"2023-06-27T12:40:03.409769+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"131'70","reported_seq":323,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-06-27T12:40:03.902859+0000","last_change":"2023-06-27T12:40:03.903614+0000","last_active":"2023-06-27T12:40:03.902859+0000","last_peered":"2023-06-27T12:40:03.902859+0000","last_clean":"2023-06-27T12:40:03.902859+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:03.902859+0000","last_undegraded":"2023-06-27T12:40:03.902859+0000","last_fullsized":"2023-06-27T12:40:03.902859+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5647152,"num_objects":9,"num_object_clones":6,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":37,"num_read_kb":37249,"num_write":45,"num_write_kb":13422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":113632094,"num_objects":164,"num_object_clones":110,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":739,"num_read_kb":520191,"num_write":829,"num_write_kb":242652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1083,"ondisk_log_size":1083,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673090,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":162976,"kb_used_data":132056,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94208864,"statfs":{"total":96636764160,"available":96469876736,"internally_reserved":0,"allocated":135225344,"data_stored":155334033,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673090,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":142280,"kb_used_data":111360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94229560,"statfs":{"total":96636764160,"available":96491069440,"internally_reserved":0,"allocated":114032640,"data_stored":132190047,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705795,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":146496,"kb_used_data":115576,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94225344,"statfs":{"total":96636764160,"available":96486752256,"internally_reserved":0,"allocated":118349824,"data_stored":134210136,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673090,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":164220,"kb_used_data":133300,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94207620,"statfs":{"total":96636764160,"available":96468602880,"internally_reserved":0,"allocated":136499200,"data_stored":161015107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:06.404 INFO:tasks.rados.rados.0.smithi138.stdout:497: done (1 left) 2023-06-27T12:40:06.404 INFO:tasks.rados.rados.0.smithi138.stdout:498: write oid 9 current snap is 67 2023-06-27T12:40:06.405 INFO:tasks.rados.rados.0.smithi138.stdout:498: seq_num 242 ranges {467294=617201,1578407=488887} 2023-06-27T12:40:06.410 INFO:tasks.rados.rados.0.smithi138.stdout:498: writing smithi138138488-9 from 467294 to 1084495 tid 1 2023-06-27T12:40:06.416 INFO:tasks.rados.rados.0.smithi138.stdout:498: writing smithi138138488-9 from 1578407 to 2067294 tid 2 2023-06-27T12:40:06.417 INFO:tasks.rados.rados.0.smithi138.stdout:499: write oid 34 current snap is 67 2023-06-27T12:40:06.417 INFO:tasks.rados.rados.0.smithi138.stdout:499: seq_num 243 ranges {484101=692450,1636889=447212} 2023-06-27T12:40:06.423 INFO:tasks.rados.rados.0.smithi138.stdout:499: writing smithi138138488-34 from 484101 to 1176551 tid 1 2023-06-27T12:40:06.428 INFO:tasks.rados.rados.0.smithi138.stdout:499: writing smithi138138488-34 from 1636889 to 2084101 tid 2 2023-06-27T12:40:06.428 INFO:tasks.rados.rados.0.smithi138.stdout:500: snap_create 2023-06-27T12:40:06.503 INFO:tasks.rados.rados.0.smithi138.stdout:499: finishing write tid 1 to smithi138138488-34 2023-06-27T12:40:06.509 INFO:tasks.rados.rados.0.smithi138.stdout:499: finishing write tid 2 to smithi138138488-34 2023-06-27T12:40:06.528 INFO:tasks.rados.rados.0.smithi138.stdout:499: finishing write tid 3 to smithi138138488-34 2023-06-27T12:40:06.533 INFO:tasks.rados.rados.0.smithi138.stdout:499: finishing write tid 4 to smithi138138488-34 2023-06-27T12:40:06.533 INFO:tasks.rados.rados.0.smithi138.stdout:499: oid 34 updating version 0 to 129 2023-06-27T12:40:06.533 INFO:tasks.rados.rados.0.smithi138.stdout:499: oid 34 version 129 is already newer than 129 2023-06-27T12:40:06.533 INFO:tasks.rados.rados.0.smithi138.stdout:499: oid 34 version 129 is already newer than 129 2023-06-27T12:40:06.534 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 129 (ObjNum 243 snap 67 seq_num 243) dirty exists 2023-06-27T12:40:06.534 INFO:tasks.rados.rados.0.smithi138.stdout:499: left oid 34 (ObjNum 243 snap 67 seq_num 243) 2023-06-27T12:40:06.560 INFO:tasks.rados.rados.0.smithi138.stdout:498: finishing write tid 1 to smithi138138488-9 2023-06-27T12:40:06.561 INFO:tasks.rados.rados.0.smithi138.stdout:498: finishing write tid 2 to smithi138138488-9 2023-06-27T12:40:06.561 INFO:tasks.rados.rados.0.smithi138.stdout:498: finishing write tid 3 to smithi138138488-9 2023-06-27T12:40:06.563 INFO:tasks.rados.rados.0.smithi138.stdout:498: finishing write tid 4 to smithi138138488-9 2023-06-27T12:40:06.563 INFO:tasks.rados.rados.0.smithi138.stdout:498: oid 9 updating version 0 to 126 2023-06-27T12:40:06.563 INFO:tasks.rados.rados.0.smithi138.stdout:498: oid 9 version 126 is already newer than 126 2023-06-27T12:40:06.564 INFO:tasks.rados.rados.0.smithi138.stdout:498: oid 9 version 126 is already newer than 126 2023-06-27T12:40:06.564 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 126 (ObjNum 242 snap 67 seq_num 242) dirty exists 2023-06-27T12:40:06.564 INFO:tasks.rados.rados.0.smithi138.stdout:498: left oid 9 (ObjNum 242 snap 67 seq_num 242) 2023-06-27T12:40:06.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:40:06.920 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:06.915+0000 7f720b59f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:06.923 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:06.919+0000 7f720b59f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:07.408 INFO:tasks.rados.rados.0.smithi138.stdout:496: done (3 left) 2023-06-27T12:40:07.409 INFO:tasks.rados.rados.0.smithi138.stdout:498: done (2 left) 2023-06-27T12:40:07.409 INFO:tasks.rados.rados.0.smithi138.stdout:499: done (1 left) 2023-06-27T12:40:07.409 INFO:tasks.rados.rados.0.smithi138.stdout:500: done (0 left) 2023-06-27T12:40:07.409 INFO:tasks.rados.rados.0.smithi138.stdout:501: write (excl) oid 44 current snap is 68 2023-06-27T12:40:07.409 INFO:tasks.rados.rados.0.smithi138.stdout:501: seq_num 244 ranges {500908=484052,1411724=689184} 2023-06-27T12:40:07.413 INFO:tasks.rados.rados.0.smithi138.stdout:501: writing smithi138138488-44 from 500908 to 984960 tid 1 2023-06-27T12:40:07.421 INFO:tasks.rados.rados.0.smithi138.stdout:501: writing smithi138138488-44 from 1411724 to 2100908 tid 2 2023-06-27T12:40:07.421 INFO:tasks.rados.rados.0.smithi138.stdout:502: write oid 49 current snap is 68 2023-06-27T12:40:07.421 INFO:tasks.rados.rados.0.smithi138.stdout:502: seq_num 245 ranges {517715=559301,1870206=247509} 2023-06-27T12:40:07.429 INFO:tasks.rados.rados.0.smithi138.stdout:502: writing smithi138138488-49 from 517715 to 1077016 tid 1 2023-06-27T12:40:07.436 INFO:tasks.rados.rados.0.smithi138.stdout:502: writing smithi138138488-49 from 1870206 to 2117715 tid 2 2023-06-27T12:40:07.436 INFO:tasks.rados.rados.0.smithi138.stdout:503: read oid 17 snap -1 2023-06-27T12:40:07.436 INFO:tasks.rados.rados.0.smithi138.stdout:503: expect (ObjNum 219 snap 49 seq_num 219) 2023-06-27T12:40:07.437 INFO:tasks.rados.rados.0.smithi138.stdout:504: read oid 34 snap -1 2023-06-27T12:40:07.437 INFO:tasks.rados.rados.0.smithi138.stdout:504: expect (ObjNum 243 snap 67 seq_num 243) 2023-06-27T12:40:07.437 INFO:tasks.rados.rados.0.smithi138.stdout:505: read oid 21 snap -1 2023-06-27T12:40:07.437 INFO:tasks.rados.rados.0.smithi138.stdout:505: expect (ObjNum 20 snap 0 seq_num 20) 2023-06-27T12:40:07.437 INFO:tasks.rados.rados.0.smithi138.stdout:506: delete oid 30 current snap is 68 2023-06-27T12:40:07.445 INFO:tasks.rados.rados.0.smithi138.stdout:506: done (5 left) 2023-06-27T12:40:07.445 INFO:tasks.rados.rados.0.smithi138.stdout:507: write (excl) oid 31 current snap is 68 2023-06-27T12:40:07.446 INFO:tasks.rados.rados.0.smithi138.stdout:507: seq_num 247 ranges {551329=709799,1870817=280512} 2023-06-27T12:40:07.452 INFO:tasks.rados.rados.0.smithi138.stdout:507: writing smithi138138488-31 from 551329 to 1261128 tid 1 2023-06-27T12:40:07.458 INFO:tasks.rados.rados.0.smithi138.stdout:507: writing smithi138138488-31 from 1870817 to 2151329 tid 2 2023-06-27T12:40:07.458 INFO:tasks.rados.rados.0.smithi138.stdout:508: write (excl) oid 13 current snap is 68 2023-06-27T12:40:07.459 INFO:tasks.rados.rados.0.smithi138.stdout:508: seq_num 248 ranges {568136=785048,1929299=238837} 2023-06-27T12:40:07.467 INFO:tasks.rados.rados.0.smithi138.stdout:508: writing smithi138138488-13 from 568136 to 1353184 tid 1 2023-06-27T12:40:07.473 INFO:tasks.rados.rados.0.smithi138.stdout:508: writing smithi138138488-13 from 1929299 to 2168136 tid 2 2023-06-27T12:40:07.473 INFO:tasks.rados.rados.0.smithi138.stdout:509: snap_create 2023-06-27T12:40:07.487 INFO:tasks.rados.rados.0.smithi138.stdout:505: expect (ObjNum 20 snap 0 seq_num 20) 2023-06-27T12:40:07.583 INFO:tasks.rados.rados.0.smithi138.stdout:501: finishing write tid 1 to smithi138138488-44 2023-06-27T12:40:07.583 INFO:tasks.rados.rados.0.smithi138.stdout:502: finishing write tid 1 to smithi138138488-49 2023-06-27T12:40:07.583 INFO:tasks.rados.rados.0.smithi138.stdout:503: expect (ObjNum 219 snap 49 seq_num 219) 2023-06-27T12:40:07.862 INFO:tasks.rados.rados.0.smithi138.stdout:502: finishing write tid 2 to smithi138138488-49 2023-06-27T12:40:07.862 INFO:tasks.rados.rados.0.smithi138.stdout:502: finishing write tid 3 to smithi138138488-49 2023-06-27T12:40:07.863 INFO:tasks.rados.rados.0.smithi138.stdout:502: finishing write tid 4 to smithi138138488-49 2023-06-27T12:40:07.863 INFO:tasks.rados.rados.0.smithi138.stdout:502: oid 49 updating version 0 to 74 2023-06-27T12:40:07.863 INFO:tasks.rados.rados.0.smithi138.stdout:502: oid 49 version 74 is already newer than 74 2023-06-27T12:40:07.863 INFO:tasks.rados.rados.0.smithi138.stdout:502: oid 49 version 74 is already newer than 74 2023-06-27T12:40:07.863 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 49 v 74 (ObjNum 245 snap 68 seq_num 245) dirty exists 2023-06-27T12:40:07.863 INFO:tasks.rados.rados.0.smithi138.stdout:502: left oid 49 (ObjNum 245 snap 68 seq_num 245) 2023-06-27T12:40:07.863 INFO:tasks.rados.rados.0.smithi138.stdout:508: finishing write tid 1 to smithi138138488-13 2023-06-27T12:40:07.863 INFO:tasks.rados.rados.0.smithi138.stdout:501: finishing write tid 2 to smithi138138488-44 2023-06-27T12:40:07.864 INFO:tasks.rados.rados.0.smithi138.stdout:501: finishing write tid 3 to smithi138138488-44 2023-06-27T12:40:07.864 INFO:tasks.rados.rados.0.smithi138.stdout:501: finishing write tid 4 to smithi138138488-44 2023-06-27T12:40:07.864 INFO:tasks.rados.rados.0.smithi138.stdout:501: oid 44 updating version 0 to 133 2023-06-27T12:40:07.864 INFO:tasks.rados.rados.0.smithi138.stdout:501: oid 44 version 133 is already newer than 130 2023-06-27T12:40:07.865 INFO:tasks.rados.rados.0.smithi138.stdout:501: oid 44 version 133 is already newer than 133 2023-06-27T12:40:07.865 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 133 (ObjNum 244 snap 68 seq_num 244) dirty exists 2023-06-27T12:40:07.865 INFO:tasks.rados.rados.0.smithi138.stdout:501: left oid 44 (ObjNum 244 snap 68 seq_num 244) 2023-06-27T12:40:07.865 INFO:tasks.rados.rados.0.smithi138.stdout:504: expect (ObjNum 243 snap 67 seq_num 243) 2023-06-27T12:40:07.982 INFO:tasks.rados.rados.0.smithi138.stdout:508: finishing write tid 2 to smithi138138488-13 2023-06-27T12:40:07.983 INFO:tasks.rados.rados.0.smithi138.stdout:508: finishing write tid 3 to smithi138138488-13 2023-06-27T12:40:07.983 INFO:tasks.rados.rados.0.smithi138.stdout:508: finishing write tid 4 to smithi138138488-13 2023-06-27T12:40:07.983 INFO:tasks.rados.rados.0.smithi138.stdout:508: oid 13 updating version 0 to 185 2023-06-27T12:40:07.983 INFO:tasks.rados.rados.0.smithi138.stdout:508: oid 13 updating version 185 to 188 2023-06-27T12:40:07.984 INFO:tasks.rados.rados.0.smithi138.stdout:508: oid 13 version 188 is already newer than 188 2023-06-27T12:40:07.984 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 188 (ObjNum 248 snap 68 seq_num 248) dirty exists 2023-06-27T12:40:07.984 INFO:tasks.rados.rados.0.smithi138.stdout:508: left oid 13 (ObjNum 248 snap 68 seq_num 248) 2023-06-27T12:40:07.985 INFO:tasks.rados.rados.0.smithi138.stdout:507: finishing write tid 1 to smithi138138488-31 2023-06-27T12:40:07.985 INFO:tasks.rados.rados.0.smithi138.stdout:507: finishing write tid 2 to smithi138138488-31 2023-06-27T12:40:07.985 INFO:tasks.rados.rados.0.smithi138.stdout:507: finishing write tid 3 to smithi138138488-31 2023-06-27T12:40:07.985 INFO:tasks.rados.rados.0.smithi138.stdout:507: finishing write tid 4 to smithi138138488-31 2023-06-27T12:40:07.986 INFO:tasks.rados.rados.0.smithi138.stdout:507: oid 31 updating version 0 to 137 2023-06-27T12:40:07.986 INFO:tasks.rados.rados.0.smithi138.stdout:507: oid 31 version 137 is already newer than 134 2023-06-27T12:40:07.986 INFO:tasks.rados.rados.0.smithi138.stdout:507: oid 31 version 137 is already newer than 137 2023-06-27T12:40:07.986 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 137 (ObjNum 247 snap 68 seq_num 247) dirty exists 2023-06-27T12:40:07.987 INFO:tasks.rados.rados.0.smithi138.stdout:507: left oid 31 (ObjNum 247 snap 68 seq_num 247) 2023-06-27T12:40:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:501: done (7 left) 2023-06-27T12:40:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:502: done (6 left) 2023-06-27T12:40:08.417 INFO:tasks.rados.rados.0.smithi138.stdout:503: done (5 left) 2023-06-27T12:40:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:504: done (4 left) 2023-06-27T12:40:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:505: done (3 left) 2023-06-27T12:40:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:507: done (2 left) 2023-06-27T12:40:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:508: done (1 left) 2023-06-27T12:40:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:509: done (0 left) 2023-06-27T12:40:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:510: write oid 18 current snap is 69 2023-06-27T12:40:08.418 INFO:tasks.rados.rados.0.smithi138.stdout:510: seq_num 249 ranges {584943=460297,1587781=597162} 2023-06-27T12:40:08.424 INFO:tasks.rados.rados.0.smithi138.stdout:510: writing smithi138138488-18 from 584943 to 1045240 tid 1 2023-06-27T12:40:08.431 INFO:tasks.rados.rados.0.smithi138.stdout:510: writing smithi138138488-18 from 1587781 to 2184943 tid 2 2023-06-27T12:40:08.432 INFO:tasks.rados.rados.0.smithi138.stdout:511: read oid 47 snap -1 2023-06-27T12:40:08.432 INFO:tasks.rados.rados.0.smithi138.stdout:511: expect (ObjNum 192 snap 38 seq_num 192) 2023-06-27T12:40:08.432 INFO:tasks.rados.rados.0.smithi138.stdout:512: snap_remove snap 62 2023-06-27T12:40:08.995 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:40:09.122 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:09.117+0000 7f5e1777d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:09.125 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:09.121+0000 7f5e1777d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:09.418 INFO:tasks.rados.rados.0.smithi138.stdout:512: done (2 left) 2023-06-27T12:40:09.419 INFO:tasks.rados.rados.0.smithi138.stdout:513: read oid 41 snap -1 2023-06-27T12:40:09.419 INFO:tasks.rados.rados.0.smithi138.stdout:513: expect (ObjNum 240 snap 64 seq_num 240) 2023-06-27T12:40:09.419 INFO:tasks.rados.rados.0.smithi138.stdout:514: delete oid 31 current snap is 69 2023-06-27T12:40:09.419 INFO:tasks.rados.rados.0.smithi138.stdout:510: finishing write tid 1 to smithi138138488-18 2023-06-27T12:40:09.419 INFO:tasks.rados.rados.0.smithi138.stdout:511: expect (ObjNum 192 snap 38 seq_num 192) 2023-06-27T12:40:09.598 INFO:tasks.rados.rados.0.smithi138.stdout:510: finishing write tid 2 to smithi138138488-18 2023-06-27T12:40:09.599 INFO:tasks.rados.rados.0.smithi138.stdout:510: finishing write tid 3 to smithi138138488-18 2023-06-27T12:40:09.599 INFO:tasks.rados.rados.0.smithi138.stdout:510: finishing write tid 4 to smithi138138488-18 2023-06-27T12:40:09.599 INFO:tasks.rados.rados.0.smithi138.stdout:510: oid 18 updating version 0 to 110 2023-06-27T12:40:09.599 INFO:tasks.rados.rados.0.smithi138.stdout:510: oid 18 version 110 is already newer than 110 2023-06-27T12:40:09.599 INFO:tasks.rados.rados.0.smithi138.stdout:510: oid 18 version 110 is already newer than 110 2023-06-27T12:40:09.599 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 110 (ObjNum 249 snap 69 seq_num 249) dirty exists 2023-06-27T12:40:09.599 INFO:tasks.rados.rados.0.smithi138.stdout:510: left oid 18 (ObjNum 249 snap 69 seq_num 249) 2023-06-27T12:40:09.600 INFO:tasks.rados.rados.0.smithi138.stdout:510: done (3 left) 2023-06-27T12:40:09.600 INFO:tasks.rados.rados.0.smithi138.stdout:511: done (2 left) 2023-06-27T12:40:09.600 INFO:tasks.rados.rados.0.smithi138.stdout:514: done (1 left) 2023-06-27T12:40:09.600 INFO:tasks.rados.rados.0.smithi138.stdout:515: write oid 40 current snap is 69 2023-06-27T12:40:09.600 INFO:tasks.rados.rados.0.smithi138.stdout:515: seq_num 251 ranges {618557=727148,2104745=113812} 2023-06-27T12:40:09.605 INFO:tasks.rados.rados.0.smithi138.stdout:515: writing smithi138138488-40 from 618557 to 1345705 tid 1 2023-06-27T12:40:09.610 INFO:tasks.rados.rados.0.smithi138.stdout:515: writing smithi138138488-40 from 2104745 to 2218557 tid 2 2023-06-27T12:40:09.611 INFO:tasks.rados.rados.0.smithi138.stdout:516: snap_create 2023-06-27T12:40:09.611 INFO:tasks.rados.rados.0.smithi138.stdout:513: expect (ObjNum 240 snap 64 seq_num 240) 2023-06-27T12:40:09.728 INFO:tasks.rados.rados.0.smithi138.stdout:515: finishing write tid 1 to smithi138138488-40 2023-06-27T12:40:09.728 INFO:tasks.rados.rados.0.smithi138.stdout:515: finishing write tid 2 to smithi138138488-40 2023-06-27T12:40:09.728 INFO:tasks.rados.rados.0.smithi138.stdout:515: finishing write tid 3 to smithi138138488-40 2023-06-27T12:40:09.728 INFO:tasks.rados.rados.0.smithi138.stdout:515: finishing write tid 4 to smithi138138488-40 2023-06-27T12:40:09.729 INFO:tasks.rados.rados.0.smithi138.stdout:515: oid 40 updating version 0 to 57 2023-06-27T12:40:09.729 INFO:tasks.rados.rados.0.smithi138.stdout:515: oid 40 version 57 is already newer than 57 2023-06-27T12:40:09.729 INFO:tasks.rados.rados.0.smithi138.stdout:515: oid 40 version 57 is already newer than 57 2023-06-27T12:40:09.729 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 57 (ObjNum 251 snap 69 seq_num 251) dirty exists 2023-06-27T12:40:09.729 INFO:tasks.rados.rados.0.smithi138.stdout:515: left oid 40 (ObjNum 251 snap 69 seq_num 251) 2023-06-27T12:40:10.397 INFO:tasks.rados.rados.0.smithi138.stdout:513: done (2 left) 2023-06-27T12:40:10.398 INFO:tasks.rados.rados.0.smithi138.stdout:515: done (1 left) 2023-06-27T12:40:10.398 INFO:tasks.rados.rados.0.smithi138.stdout:516: done (0 left) 2023-06-27T12:40:10.398 INFO:tasks.rados.rados.0.smithi138.stdout:517: snap_remove snap 60 2023-06-27T12:40:11.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:40:11.335 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:11.330+0000 7fcd630ec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:11.338 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:11.334+0000 7fcd630ec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:11.396 INFO:tasks.rados.rados.0.smithi138.stdout:517: done (0 left) 2023-06-27T12:40:11.396 INFO:tasks.rados.rados.0.smithi138.stdout:518: snap_remove snap 39 2023-06-27T12:40:11.401 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:11.401 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:11.401 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:11.529 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:11.522+0000 7f3bf5369700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:11.528+0000 7f3bf5369700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:11.728 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:11.740 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2023-06-27T12:40:11.391617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":106407438,"num_objects":174,"num_object_clones":118,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":874,"num_read_kb":546041,"num_write":998,"num_write_kb":252615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1233,"ondisk_log_size":1233,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":640432,"kb_used_data":516752,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376846928,"statfs":{"total":386547056640,"available":385891254272,"internally_reserved":0,"allocated":529154048,"data_stored":616128152,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7815024,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":25759,"num_write":32,"num_write_kb":8069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.987290"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":260,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.431584+0000","last_change":"2023-06-27T12:40:06.449344+0000","last_active":"2023-06-27T12:40:08.431584+0000","last_peered":"2023-06-27T12:40:08.431584+0000","last_clean":"2023-06-27T12:40:08.431584+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:08.431584+0000","last_undegraded":"2023-06-27T12:40:08.431584+0000","last_fullsized":"2023-06-27T12:40:08.431584+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"123'90","reported_seq":370,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.454569+0000","last_change":"2023-06-27T12:40:07.559629+0000","last_active":"2023-06-27T12:40:08.454569+0000","last_peered":"2023-06-27T12:40:08.454569+0000","last_clean":"2023-06-27T12:40:08.454569+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:08.454569+0000","last_undegraded":"2023-06-27T12:40:08.454569+0000","last_fullsized":"2023-06-27T12:40:08.454569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6907677,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":55,"num_read_kb":60674,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"140'110","reported_seq":370,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.485337+0000","last_change":"2023-06-27T12:40:06.457607+0000","last_active":"2023-06-27T12:40:08.485337+0000","last_peered":"2023-06-27T12:40:08.485337+0000","last_clean":"2023-06-27T12:40:08.485337+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:08.485337+0000","last_undegraded":"2023-06-27T12:40:08.485337+0000","last_fullsized":"2023-06-27T12:40:08.485337+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9563216,"num_objects":15,"num_object_clones":10,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":55,"num_read_kb":20450,"num_write":85,"num_write_kb":25088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"136'51","reported_seq":326,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.419197+0000","last_change":"2023-06-27T12:40:06.489593+0000","last_active":"2023-06-27T12:40:08.419197+0000","last_peered":"2023-06-27T12:40:08.419197+0000","last_clean":"2023-06-27T12:40:08.419197+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:08.419197+0000","last_undegraded":"2023-06-27T12:40:08.419197+0000","last_fullsized":"2023-06-27T12:40:08.419197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395080,"num_objects":11,"num_object_clones":7,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":54,"num_read_kb":27197,"num_write":43,"num_write_kb":12101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":297,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-06-27T12:40:07.423701+0000","last_change":"2023-06-27T12:40:06.419799+0000","last_active":"2023-06-27T12:40:07.423701+0000","last_peered":"2023-06-27T12:40:07.423701+0000","last_clean":"2023-06-27T12:40:07.423701+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:07.423701+0000","last_undegraded":"2023-06-27T12:40:07.423701+0000","last_fullsized":"2023-06-27T12:40:07.423701+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":23,"num_read_kb":23883,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"139'137","reported_seq":415,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.430272+0000","last_change":"2023-06-27T12:40:06.496118+0000","last_active":"2023-06-27T12:40:08.430272+0000","last_peered":"2023-06-27T12:40:08.430272+0000","last_clean":"2023-06-27T12:40:08.430272+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:08.430272+0000","last_undegraded":"2023-06-27T12:40:08.430272+0000","last_fullsized":"2023-06-27T12:40:08.430272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13193594,"num_objects":22,"num_object_clones":16,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":144,"num_read_kb":98218,"num_write":104,"num_write_kb":30097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":332,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.435103+0000","last_change":"2023-06-27T12:40:06.464216+0000","last_active":"2023-06-27T12:40:08.435103+0000","last_peered":"2023-06-27T12:40:08.435103+0000","last_clean":"2023-06-27T12:40:08.435103+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:08.435103+0000","last_undegraded":"2023-06-27T12:40:08.435103+0000","last_fullsized":"2023-06-27T12:40:08.435103+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":47,"num_read_kb":24452,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"117'32","reported_seq":301,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.441791+0000","last_change":"2023-06-27T12:40:06.448012+0000","last_active":"2023-06-27T12:40:08.441791+0000","last_peered":"2023-06-27T12:40:08.441791+0000","last_clean":"2023-06-27T12:40:08.441791+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:08.441791+0000","last_undegraded":"2023-06-27T12:40:08.441791+0000","last_fullsized":"2023-06-27T12:40:08.441791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3798382,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25160,"num_write":23,"num_write_kb":7301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":215,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.445781+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:08.445781+0000","last_peered":"2023-06-27T12:40:08.445781+0000","last_clean":"2023-06-27T12:40:08.445781+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:08.445781+0000","last_undegraded":"2023-06-27T12:40:08.445781+0000","last_fullsized":"2023-06-27T12:40:08.445781+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":261,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.441692+0000","last_change":"2023-06-27T12:40:06.455765+0000","last_active":"2023-06-27T12:40:08.441692+0000","last_peered":"2023-06-27T12:40:08.441692+0000","last_clean":"2023-06-27T12:40:08.441692+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:08.441692+0000","last_undegraded":"2023-06-27T12:40:08.441692+0000","last_fullsized":"2023-06-27T12:40:08.441692+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":207,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.436478+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:08.436478+0000","last_peered":"2023-06-27T12:40:08.436478+0000","last_clean":"2023-06-27T12:40:08.436478+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:08.436478+0000","last_undegraded":"2023-06-27T12:40:08.436478+0000","last_fullsized":"2023-06-27T12:40:08.436478+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"100'51","reported_seq":319,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.441646+0000","last_change":"2023-06-27T12:40:07.487472+0000","last_active":"2023-06-27T12:40:08.441646+0000","last_peered":"2023-06-27T12:40:08.441646+0000","last_clean":"2023-06-27T12:40:08.441646+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:08.441646+0000","last_undegraded":"2023-06-27T12:40:08.441646+0000","last_fullsized":"2023-06-27T12:40:08.441646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168169,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":24591,"num_write":38,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":213,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.439384+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:08.439384+0000","last_peered":"2023-06-27T12:40:08.439384+0000","last_clean":"2023-06-27T12:40:08.439384+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:08.439384+0000","last_undegraded":"2023-06-27T12:40:08.439384+0000","last_fullsized":"2023-06-27T12:40:08.439384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"139'188","reported_seq":421,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-06-27T12:40:07.579809+0000","last_change":"2023-06-27T12:40:06.465407+0000","last_active":"2023-06-27T12:40:07.579809+0000","last_peered":"2023-06-27T12:40:07.579809+0000","last_clean":"2023-06-27T12:40:07.579809+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:07.579809+0000","last_undegraded":"2023-06-27T12:40:07.579809+0000","last_fullsized":"2023-06-27T12:40:07.579809+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8974971,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":80,"num_read_kb":68414,"num_write":124,"num_write_kb":37201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":213,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.439375+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:08.439375+0000","last_peered":"2023-06-27T12:40:08.439375+0000","last_clean":"2023-06-27T12:40:08.439375+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:08.439375+0000","last_undegraded":"2023-06-27T12:40:08.439375+0000","last_fullsized":"2023-06-27T12:40:08.439375+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":338,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-06-27T12:40:07.424492+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:07.424492+0000","last_peered":"2023-06-27T12:40:07.424492+0000","last_clean":"2023-06-27T12:40:07.424492+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:07.424492+0000","last_undegraded":"2023-06-27T12:40:07.424492+0000","last_fullsized":"2023-06-27T12:40:07.424492+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"138'126","reported_seq":417,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.443400+0000","last_change":"2023-06-27T12:40:06.560946+0000","last_active":"2023-06-27T12:40:08.443400+0000","last_peered":"2023-06-27T12:40:08.443400+0000","last_clean":"2023-06-27T12:40:08.443400+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:08.443400+0000","last_undegraded":"2023-06-27T12:40:08.443400+0000","last_fullsized":"2023-06-27T12:40:08.443400+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15008618,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":120,"num_read_kb":57367,"num_write":106,"num_write_kb":30307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":207,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.420450+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:08.420450+0000","last_peered":"2023-06-27T12:40:08.420450+0000","last_clean":"2023-06-27T12:40:08.420450+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:08.420450+0000","last_undegraded":"2023-06-27T12:40:08.420450+0000","last_fullsized":"2023-06-27T12:40:08.420450+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"118'61","reported_seq":318,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-06-27T12:40:07.417469+0000","last_change":"2023-06-27T12:40:06.419527+0000","last_active":"2023-06-27T12:40:07.417469+0000","last_peered":"2023-06-27T12:40:07.417469+0000","last_clean":"2023-06-27T12:40:07.417469+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:07.417469+0000","last_undegraded":"2023-06-27T12:40:07.417469+0000","last_fullsized":"2023-06-27T12:40:07.417469+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3865610,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":30,"num_read_kb":24673,"num_write":45,"num_write_kb":13236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":207,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-06-27T12:40:07.413246+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:07.413246+0000","last_peered":"2023-06-27T12:40:07.413246+0000","last_clean":"2023-06-27T12:40:07.413246+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:07.413246+0000","last_undegraded":"2023-06-27T12:40:07.413246+0000","last_fullsized":"2023-06-27T12:40:07.413246+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'59","reported_seq":339,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.443474+0000","last_change":"2023-06-27T12:40:07.540273+0000","last_active":"2023-06-27T12:40:08.443474+0000","last_peered":"2023-06-27T12:40:08.443474+0000","last_clean":"2023-06-27T12:40:08.443474+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:08.443474+0000","last_undegraded":"2023-06-27T12:40:08.443474+0000","last_fullsized":"2023-06-27T12:40:08.443474+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12319531,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":35972,"num_write":56,"num_write_kb":16419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":305,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-06-27T12:40:07.426326+0000","last_change":"2023-06-27T12:40:06.420032+0000","last_active":"2023-06-27T12:40:07.426326+0000","last_peered":"2023-06-27T12:40:07.426326+0000","last_clean":"2023-06-27T12:40:07.426326+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:07.426326+0000","last_undegraded":"2023-06-27T12:40:07.426326+0000","last_fullsized":"2023-06-27T12:40:07.426326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":215,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.445831+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:08.445831+0000","last_peered":"2023-06-27T12:40:08.445831+0000","last_clean":"2023-06-27T12:40:08.445831+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:08.445831+0000","last_undegraded":"2023-06-27T12:40:08.445831+0000","last_fullsized":"2023-06-27T12:40:08.445831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.435563+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:08.435563+0000","last_peered":"2023-06-27T12:40:08.435563+0000","last_clean":"2023-06-27T12:40:08.435563+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:08.435563+0000","last_undegraded":"2023-06-27T12:40:08.435563+0000","last_fullsized":"2023-06-27T12:40:08.435563+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"139'74","reported_seq":337,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-06-27T12:40:08.444896+0000","last_change":"2023-06-27T12:40:07.463332+0000","last_active":"2023-06-27T12:40:08.444896+0000","last_peered":"2023-06-27T12:40:08.444896+0000","last_clean":"2023-06-27T12:40:08.444896+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:08.444896+0000","last_undegraded":"2023-06-27T12:40:08.444896+0000","last_fullsized":"2023-06-27T12:40:08.444896+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5378273,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":38,"num_read_kb":37250,"num_write":49,"num_write_kb":14211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":105817070,"num_objects":172,"num_object_clones":118,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":771,"num_read_kb":545950,"num_write":861,"num_write_kb":250721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1136,"ondisk_log_size":1136,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673095,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167872,"kb_used_data":136952,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94203968,"statfs":{"total":96636764160,"available":96464863232,"internally_reserved":0,"allocated":140238848,"data_stored":162436233,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673095,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":148304,"kb_used_data":117384,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94223536,"statfs":{"total":96636764160,"available":96484900864,"internally_reserved":0,"allocated":120201216,"data_stored":140194295,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705800,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":152836,"kb_used_data":121916,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94219004,"statfs":{"total":96636764160,"available":96480260096,"internally_reserved":0,"allocated":124841984,"data_stored":142646785,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673095,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":171420,"kb_used_data":140500,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94200420,"statfs":{"total":96636764160,"available":96461230080,"internally_reserved":0,"allocated":143872000,"data_stored":170850839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:12.398 INFO:tasks.rados.rados.0.smithi138.stdout:518: done (0 left) 2023-06-27T12:40:12.398 INFO:tasks.rados.rados.0.smithi138.stdout:519: write (excl) oid 22 current snap is 70 2023-06-27T12:40:12.398 INFO:tasks.rados.rados.0.smithi138.stdout:519: seq_num 252 ranges {635364=402397,1763227=472137} 2023-06-27T12:40:12.405 INFO:tasks.rados.rados.0.smithi138.stdout:519: writing smithi138138488-22 from 635364 to 1037761 tid 1 2023-06-27T12:40:12.413 INFO:tasks.rados.rados.0.smithi138.stdout:519: writing smithi138138488-22 from 1763227 to 2235364 tid 2 2023-06-27T12:40:12.413 INFO:tasks.rados.rados.0.smithi138.stdout:520: read oid 23 snap 50 2023-06-27T12:40:12.413 INFO:tasks.rados.rados.0.smithi138.stdout:520: expect (ObjNum 205 snap 44 seq_num 205) 2023-06-27T12:40:12.413 INFO:tasks.rados.rados.0.smithi138.stdout:521: read oid 16 snap -1 2023-06-27T12:40:12.413 INFO:tasks.rados.rados.0.smithi138.stdout:521: expect (ObjNum 175 snap 33 seq_num 175) 2023-06-27T12:40:12.414 INFO:tasks.rados.rados.0.smithi138.stdout:522: read oid 1 snap -1 2023-06-27T12:40:12.414 INFO:tasks.rados.rados.0.smithi138.stdout:522: expect (ObjNum 96 snap 7 seq_num 96) 2023-06-27T12:40:12.414 INFO:tasks.rados.rados.0.smithi138.stdout:523: delete oid 45 current snap is 70 2023-06-27T12:40:12.481 INFO:tasks.rados.rados.0.smithi138.stdout:522: expect (ObjNum 96 snap 7 seq_num 96) 2023-06-27T12:40:12.740 INFO:tasks.rados.rados.0.smithi138.stdout:521: expect (ObjNum 175 snap 33 seq_num 175) 2023-06-27T12:40:12.849 INFO:tasks.rados.rados.0.smithi138.stdout:520: expect (ObjNum 205 snap 44 seq_num 205) 2023-06-27T12:40:12.980 INFO:tasks.rados.rados.0.smithi138.stdout:519: finishing write tid 1 to smithi138138488-22 2023-06-27T12:40:12.981 INFO:tasks.rados.rados.0.smithi138.stdout:519: finishing write tid 2 to smithi138138488-22 2023-06-27T12:40:12.981 INFO:tasks.rados.rados.0.smithi138.stdout:519: finishing write tid 3 to smithi138138488-22 2023-06-27T12:40:12.981 INFO:tasks.rados.rados.0.smithi138.stdout:519: finishing write tid 4 to smithi138138488-22 2023-06-27T12:40:12.981 INFO:tasks.rados.rados.0.smithi138.stdout:519: oid 22 updating version 0 to 83 2023-06-27T12:40:12.981 INFO:tasks.rados.rados.0.smithi138.stdout:519: oid 22 updating version 83 to 86 2023-06-27T12:40:12.981 INFO:tasks.rados.rados.0.smithi138.stdout:519: oid 22 version 86 is already newer than 86 2023-06-27T12:40:12.981 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 86 (ObjNum 252 snap 70 seq_num 252) dirty exists 2023-06-27T12:40:12.981 INFO:tasks.rados.rados.0.smithi138.stdout:519: left oid 22 (ObjNum 252 snap 70 seq_num 252) 2023-06-27T12:40:12.982 INFO:tasks.rados.rados.0.smithi138.stdout:519: done (4 left) 2023-06-27T12:40:12.982 INFO:tasks.rados.rados.0.smithi138.stdout:520: done (3 left) 2023-06-27T12:40:12.982 INFO:tasks.rados.rados.0.smithi138.stdout:521: done (2 left) 2023-06-27T12:40:12.982 INFO:tasks.rados.rados.0.smithi138.stdout:522: done (1 left) 2023-06-27T12:40:12.982 INFO:tasks.rados.rados.0.smithi138.stdout:523: done (0 left) 2023-06-27T12:40:12.982 INFO:tasks.rados.rados.0.smithi138.stdout:524: write (excl) oid 18 current snap is 70 2023-06-27T12:40:12.982 INFO:tasks.rados.rados.0.smithi138.stdout:524: seq_num 254 ranges {668978=552895,1763838=505140} 2023-06-27T12:40:12.986 INFO:tasks.rados.rados.0.smithi138.stdout:524: writing smithi138138488-18 from 668978 to 1221873 tid 1 2023-06-27T12:40:12.992 INFO:tasks.rados.rados.0.smithi138.stdout:524: writing smithi138138488-18 from 1763838 to 2268978 tid 2 2023-06-27T12:40:12.992 INFO:tasks.rados.rados.0.smithi138.stdout:525: snap_remove snap 65 2023-06-27T12:40:13.412 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:40:13.436 INFO:tasks.rados.rados.0.smithi138.stdout:525: done (1 left) 2023-06-27T12:40:13.436 INFO:tasks.rados.rados.0.smithi138.stdout:526: read oid 41 snap 66 2023-06-27T12:40:13.436 INFO:tasks.rados.rados.0.smithi138.stdout:526: expect (ObjNum 240 snap 64 seq_num 240) 2023-06-27T12:40:13.436 INFO:tasks.rados.rados.0.smithi138.stdout:527: read oid 15 snap -1 2023-06-27T12:40:13.436 INFO:tasks.rados.rados.0.smithi138.stdout:527: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:40:13.437 INFO:tasks.rados.rados.0.smithi138.stdout:528: read oid 7 snap 46 2023-06-27T12:40:13.437 INFO:tasks.rados.rados.0.smithi138.stdout:528: expect (ObjNum 166 snap 26 seq_num 166) 2023-06-27T12:40:13.437 INFO:tasks.rados.rados.0.smithi138.stdout:529: write oid 26 current snap is 70 2023-06-27T12:40:13.437 INFO:tasks.rados.rados.0.smithi138.stdout:524: finishing write tid 1 to smithi138138488-18 2023-06-27T12:40:13.437 INFO:tasks.rados.rados.0.smithi138.stdout:529: seq_num 255 ranges {685785=628144,1822320=463465} 2023-06-27T12:40:13.444 INFO:tasks.rados.rados.0.smithi138.stdout:529: writing smithi138138488-26 from 685785 to 1313929 tid 1 2023-06-27T12:40:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:529: writing smithi138138488-26 from 1822320 to 2285785 tid 2 2023-06-27T12:40:13.452 INFO:tasks.rados.rados.0.smithi138.stdout:530: snap_remove snap 64 2023-06-27T12:40:13.560 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:13.555+0000 7f702b219700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:13.563 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:13.559+0000 7f702b219700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:14.443 INFO:tasks.rados.rados.0.smithi138.stdout:530: done (5 left) 2023-06-27T12:40:14.443 INFO:tasks.rados.rados.0.smithi138.stdout:531: write oid 9 current snap is 70 2023-06-27T12:40:14.443 INFO:tasks.rados.rados.0.smithi138.stdout:524: finishing write tid 2 to smithi138138488-18 2023-06-27T12:40:14.443 INFO:tasks.rados.rados.0.smithi138.stdout:531: seq_num 256 ranges {702592=703393,1880802=421790} 2023-06-27T12:40:14.450 INFO:tasks.rados.rados.0.smithi138.stdout:531: writing smithi138138488-9 from 702592 to 1405985 tid 1 2023-06-27T12:40:14.532 INFO:tasks.rados.rados.0.smithi138.stdout:531: writing smithi138138488-9 from 1880802 to 2302592 tid 2 2023-06-27T12:40:14.532 INFO:tasks.rados.rados.0.smithi138.stdout:532: read oid 48 snap 54 2023-06-27T12:40:14.532 INFO:tasks.rados.rados.0.smithi138.stdout:532: expect (ObjNum 216 snap 47 seq_num 216) 2023-06-27T12:40:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:533: delete oid 47 current snap is 70 2023-06-27T12:40:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:524: finishing write tid 3 to smithi138138488-18 2023-06-27T12:40:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:524: finishing write tid 4 to smithi138138488-18 2023-06-27T12:40:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:524: oid 18 updating version 0 to 120 2023-06-27T12:40:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:524: oid 18 version 120 is already newer than 117 2023-06-27T12:40:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:524: oid 18 version 120 is already newer than 120 2023-06-27T12:40:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 120 (ObjNum 254 snap 70 seq_num 254) dirty exists 2023-06-27T12:40:14.534 INFO:tasks.rados.rados.0.smithi138.stdout:524: left oid 18 (ObjNum 254 snap 70 seq_num 254) 2023-06-27T12:40:14.534 INFO:tasks.rados.rados.0.smithi138.stdout:526: expect (ObjNum 240 snap 64 seq_num 240) 2023-06-27T12:40:14.581 INFO:tasks.rados.rados.0.smithi138.stdout:527: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:40:14.704 INFO:tasks.rados.rados.0.smithi138.stdout:528: expect (ObjNum 166 snap 26 seq_num 166) 2023-06-27T12:40:14.917 INFO:tasks.rados.rados.0.smithi138.stdout:529: finishing write tid 1 to smithi138138488-26 2023-06-27T12:40:14.918 INFO:tasks.rados.rados.0.smithi138.stdout:529: finishing write tid 2 to smithi138138488-26 2023-06-27T12:40:14.918 INFO:tasks.rados.rados.0.smithi138.stdout:529: finishing write tid 3 to smithi138138488-26 2023-06-27T12:40:14.918 INFO:tasks.rados.rados.0.smithi138.stdout:529: finishing write tid 4 to smithi138138488-26 2023-06-27T12:40:14.918 INFO:tasks.rados.rados.0.smithi138.stdout:529: oid 26 updating version 0 to 73 2023-06-27T12:40:14.918 INFO:tasks.rados.rados.0.smithi138.stdout:529: oid 26 version 73 is already newer than 73 2023-06-27T12:40:14.918 INFO:tasks.rados.rados.0.smithi138.stdout:529: oid 26 version 73 is already newer than 73 2023-06-27T12:40:14.918 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 73 (ObjNum 255 snap 70 seq_num 255) dirty exists 2023-06-27T12:40:14.919 INFO:tasks.rados.rados.0.smithi138.stdout:529: left oid 26 (ObjNum 255 snap 70 seq_num 255) 2023-06-27T12:40:14.919 INFO:tasks.rados.rados.0.smithi138.stdout:524: done (7 left) 2023-06-27T12:40:14.919 INFO:tasks.rados.rados.0.smithi138.stdout:526: done (6 left) 2023-06-27T12:40:14.919 INFO:tasks.rados.rados.0.smithi138.stdout:527: done (5 left) 2023-06-27T12:40:14.919 INFO:tasks.rados.rados.0.smithi138.stdout:528: done (4 left) 2023-06-27T12:40:14.919 INFO:tasks.rados.rados.0.smithi138.stdout:529: done (3 left) 2023-06-27T12:40:14.919 INFO:tasks.rados.rados.0.smithi138.stdout:533: done (2 left) 2023-06-27T12:40:14.920 INFO:tasks.rados.rados.0.smithi138.stdout:534: write (excl) oid 26 current snap is 70 2023-06-27T12:40:14.920 INFO:tasks.rados.rados.0.smithi138.stdout:532: expect (ObjNum 216 snap 47 seq_num 216) 2023-06-27T12:40:15.103 INFO:tasks.rados.rados.0.smithi138.stdout:531: finishing write tid 1 to smithi138138488-9 2023-06-27T12:40:15.104 INFO:tasks.rados.rados.0.smithi138.stdout:531: finishing write tid 2 to smithi138138488-9 2023-06-27T12:40:15.104 INFO:tasks.rados.rados.0.smithi138.stdout:531: finishing write tid 3 to smithi138138488-9 2023-06-27T12:40:15.104 INFO:tasks.rados.rados.0.smithi138.stdout:531: finishing write tid 4 to smithi138138488-9 2023-06-27T12:40:15.105 INFO:tasks.rados.rados.0.smithi138.stdout:531: oid 9 updating version 0 to 140 2023-06-27T12:40:15.105 INFO:tasks.rados.rados.0.smithi138.stdout:531: oid 9 version 140 is already newer than 140 2023-06-27T12:40:15.105 INFO:tasks.rados.rados.0.smithi138.stdout:531: oid 9 version 140 is already newer than 140 2023-06-27T12:40:15.106 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 140 (ObjNum 256 snap 70 seq_num 256) dirty exists 2023-06-27T12:40:15.106 INFO:tasks.rados.rados.0.smithi138.stdout:531: left oid 9 (ObjNum 256 snap 70 seq_num 256) 2023-06-27T12:40:15.106 INFO:tasks.rados.rados.0.smithi138.stdout:534: seq_num 258 ranges {736206=453891,1881413=454793} 2023-06-27T12:40:15.110 INFO:tasks.rados.rados.0.smithi138.stdout:534: writing smithi138138488-26 from 736206 to 1190097 tid 1 2023-06-27T12:40:15.117 INFO:tasks.rados.rados.0.smithi138.stdout:534: writing smithi138138488-26 from 1881413 to 2336206 tid 2 2023-06-27T12:40:15.117 INFO:tasks.rados.rados.0.smithi138.stdout:531: done (2 left) 2023-06-27T12:40:15.118 INFO:tasks.rados.rados.0.smithi138.stdout:532: done (1 left) 2023-06-27T12:40:15.118 INFO:tasks.rados.rados.0.smithi138.stdout:535: write (excl) oid 6 current snap is 70 2023-06-27T12:40:15.118 INFO:tasks.rados.rados.0.smithi138.stdout:535: seq_num 259 ranges {753013=645493,2056248=296765} 2023-06-27T12:40:15.126 INFO:tasks.rados.rados.0.smithi138.stdout:535: writing smithi138138488-6 from 753013 to 1398506 tid 1 2023-06-27T12:40:15.131 INFO:tasks.rados.rados.0.smithi138.stdout:535: writing smithi138138488-6 from 2056248 to 2353013 tid 2 2023-06-27T12:40:15.131 INFO:tasks.rados.rados.0.smithi138.stdout:536: delete oid 31 current snap is 70 2023-06-27T12:40:15.153 INFO:tasks.rados.rados.0.smithi138.stdout:534: finishing write tid 1 to smithi138138488-26 2023-06-27T12:40:15.174 INFO:tasks.rados.rados.0.smithi138.stdout:535: finishing write tid 1 to smithi138138488-6 2023-06-27T12:40:15.197 INFO:tasks.rados.rados.0.smithi138.stdout:536: done (2 left) 2023-06-27T12:40:15.197 INFO:tasks.rados.rados.0.smithi138.stdout:537: read oid 13 snap -1 2023-06-27T12:40:15.197 INFO:tasks.rados.rados.0.smithi138.stdout:537: expect (ObjNum 248 snap 68 seq_num 248) 2023-06-27T12:40:15.197 INFO:tasks.rados.rados.0.smithi138.stdout:538: write oid 5 current snap is 70 2023-06-27T12:40:15.197 INFO:tasks.rados.rados.0.smithi138.stdout:538: seq_num 261 ranges {786627=795991,2173212=213415} 2023-06-27T12:40:15.207 INFO:tasks.rados.rados.0.smithi138.stdout:538: writing smithi138138488-5 from 786627 to 1582618 tid 1 2023-06-27T12:40:15.212 INFO:tasks.rados.rados.0.smithi138.stdout:538: writing smithi138138488-5 from 2173212 to 2386627 tid 2 2023-06-27T12:40:15.212 INFO:tasks.rados.rados.0.smithi138.stdout:539: snap_remove snap 54 2023-06-27T12:40:15.445 INFO:tasks.rados.rados.0.smithi138.stdout:539: done (4 left) 2023-06-27T12:40:15.445 INFO:tasks.rados.rados.0.smithi138.stdout:540: delete oid 16 current snap is 70 2023-06-27T12:40:15.446 INFO:tasks.rados.rados.0.smithi138.stdout:534: finishing write tid 2 to smithi138138488-26 2023-06-27T12:40:15.446 INFO:tasks.rados.rados.0.smithi138.stdout:534: finishing write tid 3 to smithi138138488-26 2023-06-27T12:40:15.446 INFO:tasks.rados.rados.0.smithi138.stdout:534: finishing write tid 4 to smithi138138488-26 2023-06-27T12:40:15.447 INFO:tasks.rados.rados.0.smithi138.stdout:534: oid 26 updating version 0 to 78 2023-06-27T12:40:15.447 INFO:tasks.rados.rados.0.smithi138.stdout:534: oid 26 version 78 is already newer than 78 2023-06-27T12:40:15.447 INFO:tasks.rados.rados.0.smithi138.stdout:534: oid 26 version 78 is already newer than 76 2023-06-27T12:40:15.447 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 78 (ObjNum 258 snap 70 seq_num 258) dirty exists 2023-06-27T12:40:15.448 INFO:tasks.rados.rados.0.smithi138.stdout:534: left oid 26 (ObjNum 258 snap 70 seq_num 258) 2023-06-27T12:40:15.448 INFO:tasks.rados.rados.0.smithi138.stdout:535: finishing write tid 2 to smithi138138488-6 2023-06-27T12:40:15.448 INFO:tasks.rados.rados.0.smithi138.stdout:537: expect (ObjNum 248 snap 68 seq_num 248) 2023-06-27T12:40:15.573 INFO:tasks.rados.rados.0.smithi138.stdout:535: finishing write tid 3 to smithi138138488-6 2023-06-27T12:40:15.574 INFO:tasks.rados.rados.0.smithi138.stdout:535: finishing write tid 4 to smithi138138488-6 2023-06-27T12:40:15.574 INFO:tasks.rados.rados.0.smithi138.stdout:535: oid 6 updating version 0 to 55 2023-06-27T12:40:15.574 INFO:tasks.rados.rados.0.smithi138.stdout:535: oid 6 version 55 is already newer than 52 2023-06-27T12:40:15.574 INFO:tasks.rados.rados.0.smithi138.stdout:535: oid 6 version 55 is already newer than 55 2023-06-27T12:40:15.574 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 55 (ObjNum 259 snap 70 seq_num 259) dirty exists 2023-06-27T12:40:15.574 INFO:tasks.rados.rados.0.smithi138.stdout:535: left oid 6 (ObjNum 259 snap 70 seq_num 259) 2023-06-27T12:40:15.575 INFO:tasks.rados.rados.0.smithi138.stdout:538: finishing write tid 1 to smithi138138488-5 2023-06-27T12:40:15.575 INFO:tasks.rados.rados.0.smithi138.stdout:538: finishing write tid 2 to smithi138138488-5 2023-06-27T12:40:15.575 INFO:tasks.rados.rados.0.smithi138.stdout:534: done (4 left) 2023-06-27T12:40:15.575 INFO:tasks.rados.rados.0.smithi138.stdout:535: done (3 left) 2023-06-27T12:40:15.575 INFO:tasks.rados.rados.0.smithi138.stdout:537: done (2 left) 2023-06-27T12:40:15.575 INFO:tasks.rados.rados.0.smithi138.stdout:540: done (1 left) 2023-06-27T12:40:15.575 INFO:tasks.rados.rados.0.smithi138.stdout:541: write (excl) oid 36 current snap is 70 2023-06-27T12:40:15.576 INFO:tasks.rados.rados.0.smithi138.stdout:538: finishing write tid 3 to smithi138138488-5 2023-06-27T12:40:15.576 INFO:tasks.rados.rados.0.smithi138.stdout:538: finishing write tid 4 to smithi138138488-5 2023-06-27T12:40:15.576 INFO:tasks.rados.rados.0.smithi138.stdout:538: oid 5 updating version 0 to 69 2023-06-27T12:40:15.576 INFO:tasks.rados.rados.0.smithi138.stdout:538: oid 5 version 69 is already newer than 69 2023-06-27T12:40:15.576 INFO:tasks.rados.rados.0.smithi138.stdout:538: oid 5 version 69 is already newer than 69 2023-06-27T12:40:15.576 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 5 v 69 (ObjNum 261 snap 70 seq_num 261) dirty exists 2023-06-27T12:40:15.576 INFO:tasks.rados.rados.0.smithi138.stdout:538: left oid 5 (ObjNum 261 snap 70 seq_num 261) 2023-06-27T12:40:15.577 INFO:tasks.rados.rados.0.smithi138.stdout:541: seq_num 263 ranges {420241=546489,1373823=544414} 2023-06-27T12:40:15.578 INFO:tasks.rados.rados.0.smithi138.stdout:541: writing smithi138138488-36 from 420241 to 966730 tid 1 2023-06-27T12:40:15.584 INFO:tasks.rados.rados.0.smithi138.stdout:541: writing smithi138138488-36 from 1373823 to 1918237 tid 2 2023-06-27T12:40:15.584 INFO:tasks.rados.rados.0.smithi138.stdout:538: done (1 left) 2023-06-27T12:40:15.584 INFO:tasks.rados.rados.0.smithi138.stdout:542: write oid 31 current snap is 70 2023-06-27T12:40:15.585 INFO:tasks.rados.rados.0.smithi138.stdout:542: seq_num 264 ranges {437048=621738,1832305=404425} 2023-06-27T12:40:15.592 INFO:tasks.rados.rados.0.smithi138.stdout:542: writing smithi138138488-31 from 437048 to 1058786 tid 1 2023-06-27T12:40:15.600 INFO:tasks.rados.rados.0.smithi138.stdout:542: writing smithi138138488-31 from 1832305 to 2236730 tid 2 2023-06-27T12:40:15.600 INFO:tasks.rados.rados.0.smithi138.stdout:543: write oid 21 current snap is 70 2023-06-27T12:40:15.601 INFO:tasks.rados.rados.0.smithi138.stdout:543: seq_num 265 ranges {453855=696987,1890787=548083} 2023-06-27T12:40:15.609 INFO:tasks.rados.rados.0.smithi138.stdout:543: writing smithi138138488-21 from 453855 to 1150842 tid 1 2023-06-27T12:40:15.616 INFO:tasks.rados.rados.0.smithi138.stdout:543: writing smithi138138488-21 from 1890787 to 2438870 tid 2 2023-06-27T12:40:15.616 INFO:tasks.rados.rados.0.smithi138.stdout:544: delete oid 45 current snap is 70 2023-06-27T12:40:15.620 INFO:tasks.rados.rados.0.smithi138.stdout:544: done (3 left) 2023-06-27T12:40:15.620 INFO:tasks.rados.rados.0.smithi138.stdout:545: write oid 7 current snap is 70 2023-06-27T12:40:15.620 INFO:tasks.rados.rados.0.smithi138.stdout:545: seq_num 267 ranges {487469=563838,1607751=551752} 2023-06-27T12:40:15.626 INFO:tasks.rados.rados.0.smithi138.stdout:545: writing smithi138138488-7 from 487469 to 1051307 tid 1 2023-06-27T12:40:15.633 INFO:tasks.rados.rados.0.smithi138.stdout:545: writing smithi138138488-7 from 1607751 to 2159503 tid 2 2023-06-27T12:40:15.633 INFO:tasks.rados.rados.0.smithi138.stdout:546: read oid 47 snap 63 2023-06-27T12:40:15.633 INFO:tasks.rados.rados.0.smithi138.stdout:546: expect (ObjNum 192 snap 38 seq_num 192) 2023-06-27T12:40:15.633 INFO:tasks.rados.rados.0.smithi138.stdout:547: write oid 41 current snap is 70 2023-06-27T12:40:15.633 INFO:tasks.rados.rados.0.smithi138.stdout:547: seq_num 268 ranges {504276=639087,1666233=695410} 2023-06-27T12:40:15.635 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:40:15.639 INFO:tasks.rados.rados.0.smithi138.stdout:547: writing smithi138138488-41 from 504276 to 1143363 tid 1 2023-06-27T12:40:15.646 INFO:tasks.rados.rados.0.smithi138.stdout:547: writing smithi138138488-41 from 1666233 to 2361643 tid 2 2023-06-27T12:40:15.647 INFO:tasks.rados.rados.0.smithi138.stdout:548: snap_remove snap 66 2023-06-27T12:40:15.782 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:15.777+0000 7f515021e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:15.786 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:15.782+0000 7f515021e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:16.454 INFO:tasks.rados.rados.0.smithi138.stdout:548: done (6 left) 2023-06-27T12:40:16.455 INFO:tasks.rados.rados.0.smithi138.stdout:549: write oid 11 current snap is 70 2023-06-27T12:40:16.455 INFO:tasks.rados.rados.0.smithi138.stdout:542: finishing write tid 1 to smithi138138488-31 2023-06-27T12:40:16.455 INFO:tasks.rados.rados.0.smithi138.stdout:549: seq_num 269 ranges {521083=714336,1724715=555421} 2023-06-27T12:40:16.463 INFO:tasks.rados.rados.0.smithi138.stdout:549: writing smithi138138488-11 from 521083 to 1235419 tid 1 2023-06-27T12:40:16.472 INFO:tasks.rados.rados.0.smithi138.stdout:549: writing smithi138138488-11 from 1724715 to 2280136 tid 2 2023-06-27T12:40:16.472 INFO:tasks.rados.rados.0.smithi138.stdout:550: write (excl) oid 9 current snap is 70 2023-06-27T12:40:16.472 INFO:tasks.rados.rados.0.smithi138.stdout:550: seq_num 270 ranges {537890=789585,1783197=699079} 2023-06-27T12:40:16.482 INFO:tasks.rados.rados.0.smithi138.stdout:550: writing smithi138138488-9 from 537890 to 1327475 tid 1 2023-06-27T12:40:16.491 INFO:tasks.rados.rados.0.smithi138.stdout:550: writing smithi138138488-9 from 1783197 to 2482276 tid 2 2023-06-27T12:40:16.491 INFO:tasks.rados.rados.0.smithi138.stdout:551: snap_remove snap 46 2023-06-27T12:40:16.742 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:16.742 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:16.743 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:16.874 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:16.867+0000 7effde849700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:16.874+0000 7effde849700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:17.073 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:17.074 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:17.086 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2023-06-27T12:40:16.449760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98844321,"num_objects":180,"num_object_clones":124,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":21,"num_read":911,"num_read_kb":581871,"num_write":1014,"num_write_kb":256394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1285,"ondisk_log_size":1285,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":641292,"kb_used_data":517612,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376846068,"statfs":{"total":386547056640,"available":385890373632,"internally_reserved":0,"allocated":530034688,"data_stored":618163489,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7563117,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":37,"num_read_kb":35830,"num_write":16,"num_write_kb":3779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.056862"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":272,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.450421+0000","last_change":"2023-06-27T12:40:13.453300+0000","last_active":"2023-06-27T12:40:13.450421+0000","last_peered":"2023-06-27T12:40:13.450421+0000","last_clean":"2023-06-27T12:40:13.450421+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:13.450421+0000","last_undegraded":"2023-06-27T12:40:13.450421+0000","last_fullsized":"2023-06-27T12:40:13.450421+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"123'90","reported_seq":384,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.444693+0000","last_change":"2023-06-27T12:40:13.459733+0000","last_active":"2023-06-27T12:40:13.444693+0000","last_peered":"2023-06-27T12:40:13.444693+0000","last_clean":"2023-06-27T12:40:13.444693+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:13.444693+0000","last_undegraded":"2023-06-27T12:40:13.444693+0000","last_fullsized":"2023-06-27T12:40:13.444693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6907677,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":55,"num_read_kb":60674,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"144'120","reported_seq":383,"reported_epoch":145,"state":"active+clean+snaptrim","last_fresh":"2023-06-27T12:40:13.912721+0000","last_change":"2023-06-27T12:40:12.425716+0000","last_active":"2023-06-27T12:40:13.912721+0000","last_peered":"2023-06-27T12:40:13.912721+0000","last_clean":"2023-06-27T12:40:13.912721+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:13.912721+0000","last_undegraded":"2023-06-27T12:40:13.912721+0000","last_fullsized":"2023-06-27T12:40:13.912721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9647251,"num_objects":16,"num_object_clones":11,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":20451,"num_write":89,"num_write_kb":26122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"141'57","reported_seq":343,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.452534+0000","last_change":"2023-06-27T12:40:13.452644+0000","last_active":"2023-06-27T12:40:13.452534+0000","last_peered":"2023-06-27T12:40:13.452534+0000","last_clean":"2023-06-27T12:40:13.452534+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:13.452534+0000","last_undegraded":"2023-06-27T12:40:13.452534+0000","last_fullsized":"2023-06-27T12:40:13.452534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563150,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":55,"num_read_kb":27198,"num_write":47,"num_write_kb":12924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":314,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.499772+0000","last_change":"2023-06-27T12:40:13.488654+0000","last_active":"2023-06-27T12:40:13.499772+0000","last_peered":"2023-06-27T12:40:13.499772+0000","last_clean":"2023-06-27T12:40:13.499772+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:13.499772+0000","last_undegraded":"2023-06-27T12:40:13.499772+0000","last_fullsized":"2023-06-27T12:40:13.499772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"141'139","reported_seq":430,"reported_epoch":145,"state":"active+clean+snaptrim","last_fresh":"2023-06-27T12:40:13.454795+0000","last_change":"2023-06-27T12:40:13.454795+0000","last_active":"2023-06-27T12:40:13.454795+0000","last_peered":"2023-06-27T12:40:13.454795+0000","last_clean":"2023-06-27T12:40:13.454795+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:13.454795+0000","last_undegraded":"2023-06-27T12:40:13.454795+0000","last_fullsized":"2023-06-27T12:40:13.454795+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11042265,"num_objects":23,"num_object_clones":17,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":152,"num_read_kb":103227,"num_write":104,"num_write_kb":30097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"83'75","reported_seq":347,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.504599+0000","last_change":"2023-06-27T12:40:13.451191+0000","last_active":"2023-06-27T12:40:13.504599+0000","last_peered":"2023-06-27T12:40:13.504599+0000","last_clean":"2023-06-27T12:40:13.504599+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:13.504599+0000","last_undegraded":"2023-06-27T12:40:13.504599+0000","last_fullsized":"2023-06-27T12:40:13.504599+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5848836,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":54,"num_read_kb":30116,"num_write":55,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"117'32","reported_seq":314,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.911141+0000","last_change":"2023-06-27T12:40:13.915946+0000","last_active":"2023-06-27T12:40:13.911141+0000","last_peered":"2023-06-27T12:40:13.911141+0000","last_clean":"2023-06-27T12:40:13.911141+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:13.911141+0000","last_undegraded":"2023-06-27T12:40:13.911141+0000","last_fullsized":"2023-06-27T12:40:13.911141+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3798382,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25160,"num_write":23,"num_write_kb":7301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":225,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.449854+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:13.449854+0000","last_peered":"2023-06-27T12:40:13.449854+0000","last_clean":"2023-06-27T12:40:13.449854+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:13.449854+0000","last_undegraded":"2023-06-27T12:40:13.449854+0000","last_fullsized":"2023-06-27T12:40:13.449854+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":273,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.915713+0000","last_change":"2023-06-27T12:40:13.928920+0000","last_active":"2023-06-27T12:40:13.915713+0000","last_peered":"2023-06-27T12:40:13.915713+0000","last_clean":"2023-06-27T12:40:13.915713+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:13.915713+0000","last_undegraded":"2023-06-27T12:40:13.915713+0000","last_fullsized":"2023-06-27T12:40:13.915713+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":215,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.445429+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:13.445429+0000","last_peered":"2023-06-27T12:40:13.445429+0000","last_clean":"2023-06-27T12:40:13.445429+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:13.445429+0000","last_undegraded":"2023-06-27T12:40:13.445429+0000","last_fullsized":"2023-06-27T12:40:13.445429+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"100'51","reported_seq":335,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.450000+0000","last_change":"2023-06-27T12:40:13.460735+0000","last_active":"2023-06-27T12:40:13.450000+0000","last_peered":"2023-06-27T12:40:13.450000+0000","last_clean":"2023-06-27T12:40:13.450000+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:13.450000+0000","last_undegraded":"2023-06-27T12:40:13.450000+0000","last_fullsized":"2023-06-27T12:40:13.450000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168169,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":32,"num_read_kb":31321,"num_write":38,"num_write_kb":10517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":221,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.910472+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:13.910472+0000","last_peered":"2023-06-27T12:40:13.910472+0000","last_clean":"2023-06-27T12:40:13.910472+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:13.910472+0000","last_undegraded":"2023-06-27T12:40:13.910472+0000","last_fullsized":"2023-06-27T12:40:13.910472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"144'196","reported_seq":437,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.465670+0000","last_change":"2023-06-27T12:40:13.533674+0000","last_active":"2023-06-27T12:40:13.465670+0000","last_peered":"2023-06-27T12:40:13.465670+0000","last_clean":"2023-06-27T12:40:13.465670+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:13.465670+0000","last_undegraded":"2023-06-27T12:40:13.465670+0000","last_fullsized":"2023-06-27T12:40:13.465670+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5075747,"num_objects":23,"num_object_clones":18,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":81,"num_read_kb":68414,"num_write":124,"num_write_kb":37201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":221,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.909247+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:13.909247+0000","last_peered":"2023-06-27T12:40:13.909247+0000","last_clean":"2023-06-27T12:40:13.909247+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:13.909247+0000","last_undegraded":"2023-06-27T12:40:13.909247+0000","last_fullsized":"2023-06-27T12:40:13.909247+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":348,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.462098+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:13.462098+0000","last_peered":"2023-06-27T12:40:13.462098+0000","last_clean":"2023-06-27T12:40:13.462098+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:13.462098+0000","last_undegraded":"2023-06-27T12:40:13.462098+0000","last_fullsized":"2023-06-27T12:40:13.462098+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"138'126","reported_seq":431,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.913135+0000","last_change":"2023-06-27T12:40:13.939870+0000","last_active":"2023-06-27T12:40:13.913135+0000","last_peered":"2023-06-27T12:40:13.913135+0000","last_clean":"2023-06-27T12:40:13.913135+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:13.913135+0000","last_undegraded":"2023-06-27T12:40:13.913135+0000","last_fullsized":"2023-06-27T12:40:13.913135+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15008618,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":127,"num_read_kb":64216,"num_write":106,"num_write_kb":30307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":215,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.456099+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:13.456099+0000","last_peered":"2023-06-27T12:40:13.456099+0000","last_clean":"2023-06-27T12:40:13.456099+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:13.456099+0000","last_undegraded":"2023-06-27T12:40:13.456099+0000","last_fullsized":"2023-06-27T12:40:13.456099+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"118'61","reported_seq":332,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.466120+0000","last_change":"2023-06-27T12:40:13.471745+0000","last_active":"2023-06-27T12:40:13.466120+0000","last_peered":"2023-06-27T12:40:13.466120+0000","last_clean":"2023-06-27T12:40:13.466120+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:13.466120+0000","last_undegraded":"2023-06-27T12:40:13.466120+0000","last_fullsized":"2023-06-27T12:40:13.466120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3865610,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":30,"num_read_kb":24673,"num_write":45,"num_write_kb":13236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":217,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.467062+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:13.467062+0000","last_peered":"2023-06-27T12:40:13.467062+0000","last_clean":"2023-06-27T12:40:13.467062+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:13.467062+0000","last_undegraded":"2023-06-27T12:40:13.467062+0000","last_fullsized":"2023-06-27T12:40:13.467062+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"145'73","reported_seq":361,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.498270+0000","last_change":"2023-06-27T12:40:13.457019+0000","last_active":"2023-06-27T12:40:13.498270+0000","last_peered":"2023-06-27T12:40:13.498270+0000","last_clean":"2023-06-27T12:40:13.498270+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:13.498270+0000","last_undegraded":"2023-06-27T12:40:13.498270+0000","last_fullsized":"2023-06-27T12:40:13.498270+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11580056,"num_objects":11,"num_object_clones":7,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":42,"num_read_kb":42851,"num_write":60,"num_write_kb":17486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":320,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.469538+0000","last_change":"2023-06-27T12:40:13.472262+0000","last_active":"2023-06-27T12:40:13.469538+0000","last_peered":"2023-06-27T12:40:13.469538+0000","last_clean":"2023-06-27T12:40:13.469538+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:13.469538+0000","last_undegraded":"2023-06-27T12:40:13.469538+0000","last_fullsized":"2023-06-27T12:40:13.469538+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":225,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.441817+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:13.441817+0000","last_peered":"2023-06-27T12:40:13.441817+0000","last_clean":"2023-06-27T12:40:13.441817+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:13.441817+0000","last_undegraded":"2023-06-27T12:40:13.441817+0000","last_fullsized":"2023-06-27T12:40:13.441817+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":221,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.914298+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:13.914298+0000","last_peered":"2023-06-27T12:40:13.914298+0000","last_clean":"2023-06-27T12:40:13.914298+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:13.914298+0000","last_undegraded":"2023-06-27T12:40:13.914298+0000","last_fullsized":"2023-06-27T12:40:13.914298+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"144'86","reported_seq":356,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-06-27T12:40:13.449883+0000","last_change":"2023-06-27T12:40:13.525821+0000","last_active":"2023-06-27T12:40:13.449883+0000","last_peered":"2023-06-27T12:40:13.449883+0000","last_clean":"2023-06-27T12:40:13.449883+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:13.449883+0000","last_undegraded":"2023-06-27T12:40:13.449883+0000","last_fullsized":"2023-06-27T12:40:13.449883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4353079,"num_objects":11,"num_object_clones":8,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":40,"num_read_kb":37251,"num_write":53,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":98253953,"num_objects":178,"num_object_clones":124,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":21,"num_read":808,"num_read_kb":581780,"num_write":877,"num_write_kb":254500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1188,"ondisk_log_size":1188,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673100,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":168968,"kb_used_data":138048,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94202872,"statfs":{"total":96636764160,"available":96463740928,"internally_reserved":0,"allocated":141361152,"data_stored":164396240,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673100,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":150276,"kb_used_data":119356,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94221564,"statfs":{"total":96636764160,"available":96482881536,"internally_reserved":0,"allocated":122220544,"data_stored":143002445,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705805,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":152256,"kb_used_data":121336,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94219584,"statfs":{"total":96636764160,"available":96480854016,"internally_reserved":0,"allocated":124248064,"data_stored":142140105,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673100,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":169792,"kb_used_data":138872,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94202048,"statfs":{"total":96636764160,"available":96462897152,"internally_reserved":0,"allocated":142204928,"data_stored":168624699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:17.456 INFO:tasks.rados.rados.0.smithi138.stdout:551: done (8 left) 2023-06-27T12:40:17.456 INFO:tasks.rados.rados.0.smithi138.stdout:552: write (excl) oid 35 current snap is 70 2023-06-27T12:40:17.456 INFO:tasks.rados.rados.0.smithi138.stdout:552: seq_num 271 ranges {554697=464834,1725326=559090} 2023-06-27T12:40:17.461 INFO:tasks.rados.rados.0.smithi138.stdout:552: writing smithi138138488-35 from 554697 to 1019531 tid 1 2023-06-27T12:40:17.470 INFO:tasks.rados.rados.0.smithi138.stdout:552: writing smithi138138488-35 from 1725326 to 2284416 tid 2 2023-06-27T12:40:17.470 INFO:tasks.rados.rados.0.smithi138.stdout:553: read oid 46 snap 58 2023-06-27T12:40:17.470 INFO:tasks.rados.rados.0.smithi138.stdout:553: expect deleted 2023-06-27T12:40:17.470 INFO:tasks.rados.rados.0.smithi138.stdout:554: read oid 45 snap -1 2023-06-27T12:40:17.471 INFO:tasks.rados.rados.0.smithi138.stdout:554: expect deleted 2023-06-27T12:40:17.471 INFO:tasks.rados.rados.0.smithi138.stdout:555: delete oid 4 current snap is 70 2023-06-27T12:40:17.471 INFO:tasks.rados.rados.0.smithi138.stdout:542: finishing write tid 2 to smithi138138488-31 2023-06-27T12:40:17.471 INFO:tasks.rados.rados.0.smithi138.stdout:542: finishing write tid 3 to smithi138138488-31 2023-06-27T12:40:17.472 INFO:tasks.rados.rados.0.smithi138.stdout:542: finishing write tid 4 to smithi138138488-31 2023-06-27T12:40:17.472 INFO:tasks.rados.rados.0.smithi138.stdout:542: oid 31 updating version 0 to 167 2023-06-27T12:40:17.472 INFO:tasks.rados.rados.0.smithi138.stdout:542: oid 31 version 167 is already newer than 167 2023-06-27T12:40:17.472 INFO:tasks.rados.rados.0.smithi138.stdout:542: oid 31 version 167 is already newer than 167 2023-06-27T12:40:17.473 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 167 (ObjNum 264 snap 70 seq_num 264) dirty exists 2023-06-27T12:40:17.473 INFO:tasks.rados.rados.0.smithi138.stdout:542: left oid 31 (ObjNum 264 snap 70 seq_num 264) 2023-06-27T12:40:17.473 INFO:tasks.rados.rados.0.smithi138.stdout:545: finishing write tid 1 to smithi138138488-7 2023-06-27T12:40:17.473 INFO:tasks.rados.rados.0.smithi138.stdout:541: finishing write tid 1 to smithi138138488-36 2023-06-27T12:40:17.473 INFO:tasks.rados.rados.0.smithi138.stdout:543: finishing write tid 1 to smithi138138488-21 2023-06-27T12:40:17.473 INFO:tasks.rados.rados.0.smithi138.stdout:545: finishing write tid 2 to smithi138138488-7 2023-06-27T12:40:17.473 INFO:tasks.rados.rados.0.smithi138.stdout:543: finishing write tid 2 to smithi138138488-21 2023-06-27T12:40:17.474 INFO:tasks.rados.rados.0.smithi138.stdout:543: finishing write tid 3 to smithi138138488-21 2023-06-27T12:40:17.474 INFO:tasks.rados.rados.0.smithi138.stdout:543: finishing write tid 4 to smithi138138488-21 2023-06-27T12:40:17.474 INFO:tasks.rados.rados.0.smithi138.stdout:543: oid 21 updating version 0 to 146 2023-06-27T12:40:17.474 INFO:tasks.rados.rados.0.smithi138.stdout:543: oid 21 version 146 is already newer than 146 2023-06-27T12:40:17.474 INFO:tasks.rados.rados.0.smithi138.stdout:543: oid 21 version 146 is already newer than 146 2023-06-27T12:40:17.474 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 21 v 146 (ObjNum 265 snap 70 seq_num 265) dirty exists 2023-06-27T12:40:17.474 INFO:tasks.rados.rados.0.smithi138.stdout:543: left oid 21 (ObjNum 265 snap 70 seq_num 265) 2023-06-27T12:40:17.474 INFO:tasks.rados.rados.0.smithi138.stdout:545: finishing write tid 3 to smithi138138488-7 2023-06-27T12:40:17.475 INFO:tasks.rados.rados.0.smithi138.stdout:547: finishing write tid 1 to smithi138138488-41 2023-06-27T12:40:17.475 INFO:tasks.rados.rados.0.smithi138.stdout:545: finishing write tid 4 to smithi138138488-7 2023-06-27T12:40:17.475 INFO:tasks.rados.rados.0.smithi138.stdout:545: oid 7 updating version 0 to 87 2023-06-27T12:40:17.475 INFO:tasks.rados.rados.0.smithi138.stdout:545: oid 7 version 87 is already newer than 87 2023-06-27T12:40:17.475 INFO:tasks.rados.rados.0.smithi138.stdout:545: oid 7 version 87 is already newer than 87 2023-06-27T12:40:17.475 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 87 (ObjNum 267 snap 70 seq_num 267) dirty exists 2023-06-27T12:40:17.475 INFO:tasks.rados.rados.0.smithi138.stdout:545: left oid 7 (ObjNum 267 snap 70 seq_num 267) 2023-06-27T12:40:17.475 INFO:tasks.rados.rados.0.smithi138.stdout:546: expect (ObjNum 192 snap 38 seq_num 192) 2023-06-27T12:40:17.655 INFO:tasks.rados.rados.0.smithi138.stdout:547: finishing write tid 2 to smithi138138488-41 2023-06-27T12:40:17.655 INFO:tasks.rados.rados.0.smithi138.stdout:547: finishing write tid 3 to smithi138138488-41 2023-06-27T12:40:17.655 INFO:tasks.rados.rados.0.smithi138.stdout:547: finishing write tid 4 to smithi138138488-41 2023-06-27T12:40:17.655 INFO:tasks.rados.rados.0.smithi138.stdout:547: oid 41 updating version 0 to 86 2023-06-27T12:40:17.656 INFO:tasks.rados.rados.0.smithi138.stdout:547: oid 41 version 86 is already newer than 86 2023-06-27T12:40:17.656 INFO:tasks.rados.rados.0.smithi138.stdout:547: oid 41 version 86 is already newer than 86 2023-06-27T12:40:17.656 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 41 v 86 (ObjNum 268 snap 70 seq_num 268) dirty exists 2023-06-27T12:40:17.656 INFO:tasks.rados.rados.0.smithi138.stdout:547: left oid 41 (ObjNum 268 snap 70 seq_num 268) 2023-06-27T12:40:17.656 INFO:tasks.rados.rados.0.smithi138.stdout:541: finishing write tid 2 to smithi138138488-36 2023-06-27T12:40:17.656 INFO:tasks.rados.rados.0.smithi138.stdout:541: finishing write tid 3 to smithi138138488-36 2023-06-27T12:40:17.656 INFO:tasks.rados.rados.0.smithi138.stdout:541: finishing write tid 4 to smithi138138488-36 2023-06-27T12:40:17.656 INFO:tasks.rados.rados.0.smithi138.stdout:541: oid 36 updating version 0 to 130 2023-06-27T12:40:17.657 INFO:tasks.rados.rados.0.smithi138.stdout:541: oid 36 version 130 is already newer than 130 2023-06-27T12:40:17.657 INFO:tasks.rados.rados.0.smithi138.stdout:541: oid 36 version 130 is already newer than 127 2023-06-27T12:40:17.657 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 130 (ObjNum 263 snap 70 seq_num 263) dirty exists 2023-06-27T12:40:17.657 INFO:tasks.rados.rados.0.smithi138.stdout:541: left oid 36 (ObjNum 263 snap 70 seq_num 263) 2023-06-27T12:40:17.657 INFO:tasks.rados.rados.0.smithi138.stdout:541: done (11 left) 2023-06-27T12:40:17.657 INFO:tasks.rados.rados.0.smithi138.stdout:542: done (10 left) 2023-06-27T12:40:17.657 INFO:tasks.rados.rados.0.smithi138.stdout:543: done (9 left) 2023-06-27T12:40:17.658 INFO:tasks.rados.rados.0.smithi138.stdout:545: done (8 left) 2023-06-27T12:40:17.658 INFO:tasks.rados.rados.0.smithi138.stdout:546: done (7 left) 2023-06-27T12:40:17.658 INFO:tasks.rados.rados.0.smithi138.stdout:547: done (6 left) 2023-06-27T12:40:17.658 INFO:tasks.rados.rados.0.smithi138.stdout:555: done (5 left) 2023-06-27T12:40:17.658 INFO:tasks.rados.rados.0.smithi138.stdout:556: snap_remove snap 57 2023-06-27T12:40:17.883 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:40:18.010 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:18.005+0000 7fdb50023700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:18.013 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:18.009+0000 7fdb50023700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:18.463 INFO:tasks.rados.rados.0.smithi138.stdout:556: done (5 left) 2023-06-27T12:40:18.463 INFO:tasks.rados.rados.0.smithi138.stdout:557: read oid 42 snap -1 2023-06-27T12:40:18.463 INFO:tasks.rados.rados.0.smithi138.stdout:557: expect deleted 2023-06-27T12:40:18.463 INFO:tasks.rados.rados.0.smithi138.stdout:558: snap_remove snap 68 2023-06-27T12:40:19.465 INFO:tasks.rados.rados.0.smithi138.stdout:558: done (6 left) 2023-06-27T12:40:19.465 INFO:tasks.rados.rados.0.smithi138.stdout:559: read oid 24 snap -1 2023-06-27T12:40:19.465 INFO:tasks.rados.rados.0.smithi138.stdout:559: expect (ObjNum 233 snap 58 seq_num 233) 2023-06-27T12:40:19.465 INFO:tasks.rados.rados.0.smithi138.stdout:560: write oid 4 current snap is 70 2023-06-27T12:40:19.465 INFO:tasks.rados.rados.0.smithi138.stdout:549: finishing write tid 1 to smithi138138488-11 2023-06-27T12:40:19.465 INFO:tasks.rados.rados.0.smithi138.stdout:560: seq_num 273 ranges {588311=615332,1842290=562759} 2023-06-27T12:40:19.471 INFO:tasks.rados.rados.0.smithi138.stdout:560: writing smithi138138488-4 from 588311 to 1203643 tid 1 2023-06-27T12:40:19.478 INFO:tasks.rados.rados.0.smithi138.stdout:560: writing smithi138138488-4 from 1842290 to 2405049 tid 2 2023-06-27T12:40:19.478 INFO:tasks.rados.rados.0.smithi138.stdout:561: snap_remove snap 53 2023-06-27T12:40:20.084 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:40:20.214 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:20.209+0000 7f5289b18700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:20.217 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:20.213+0000 7f5289b18700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:20.400 INFO:tasks.rados.rados.0.smithi138.stdout:561: done (8 left) 2023-06-27T12:40:20.400 INFO:tasks.rados.rados.0.smithi138.stdout:562: snap_create 2023-06-27T12:40:20.401 INFO:tasks.rados.rados.0.smithi138.stdout:549: finishing write tid 2 to smithi138138488-11 2023-06-27T12:40:20.401 INFO:tasks.rados.rados.0.smithi138.stdout:549: finishing write tid 3 to smithi138138488-11 2023-06-27T12:40:20.401 INFO:tasks.rados.rados.0.smithi138.stdout:549: finishing write tid 4 to smithi138138488-11 2023-06-27T12:40:20.401 INFO:tasks.rados.rados.0.smithi138.stdout:549: oid 11 updating version 0 to 138 2023-06-27T12:40:20.401 INFO:tasks.rados.rados.0.smithi138.stdout:549: oid 11 version 138 is already newer than 138 2023-06-27T12:40:20.401 INFO:tasks.rados.rados.0.smithi138.stdout:549: oid 11 version 138 is already newer than 138 2023-06-27T12:40:20.401 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 11 v 138 (ObjNum 269 snap 70 seq_num 269) dirty exists 2023-06-27T12:40:20.402 INFO:tasks.rados.rados.0.smithi138.stdout:549: left oid 11 (ObjNum 269 snap 70 seq_num 269) 2023-06-27T12:40:20.402 INFO:tasks.rados.rados.0.smithi138.stdout:550: finishing write tid 1 to smithi138138488-9 2023-06-27T12:40:20.402 INFO:tasks.rados.rados.0.smithi138.stdout:550: finishing write tid 2 to smithi138138488-9 2023-06-27T12:40:20.402 INFO:tasks.rados.rados.0.smithi138.stdout:550: finishing write tid 3 to smithi138138488-9 2023-06-27T12:40:20.402 INFO:tasks.rados.rados.0.smithi138.stdout:550: finishing write tid 4 to smithi138138488-9 2023-06-27T12:40:20.402 INFO:tasks.rados.rados.0.smithi138.stdout:550: oid 9 updating version 0 to 151 2023-06-27T12:40:20.402 INFO:tasks.rados.rados.0.smithi138.stdout:550: oid 9 updating version 151 to 153 2023-06-27T12:40:20.402 INFO:tasks.rados.rados.0.smithi138.stdout:550: oid 9 version 153 is already newer than 153 2023-06-27T12:40:20.403 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 153 (ObjNum 270 snap 70 seq_num 270) dirty exists 2023-06-27T12:40:20.403 INFO:tasks.rados.rados.0.smithi138.stdout:550: left oid 9 (ObjNum 270 snap 70 seq_num 270) 2023-06-27T12:40:20.403 INFO:tasks.rados.rados.0.smithi138.stdout:552: finishing write tid 1 to smithi138138488-35 2023-06-27T12:40:20.403 INFO:tasks.rados.rados.0.smithi138.stdout:552: finishing write tid 2 to smithi138138488-35 2023-06-27T12:40:20.403 INFO:tasks.rados.rados.0.smithi138.stdout:552: finishing write tid 3 to smithi138138488-35 2023-06-27T12:40:20.403 INFO:tasks.rados.rados.0.smithi138.stdout:552: finishing write tid 4 to smithi138138488-35 2023-06-27T12:40:20.403 INFO:tasks.rados.rados.0.smithi138.stdout:552: oid 35 updating version 0 to 214 2023-06-27T12:40:20.404 INFO:tasks.rados.rados.0.smithi138.stdout:552: oid 35 updating version 214 to 216 2023-06-27T12:40:20.404 INFO:tasks.rados.rados.0.smithi138.stdout:552: oid 35 version 216 is already newer than 216 2023-06-27T12:40:20.404 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 216 (ObjNum 271 snap 70 seq_num 271) dirty exists 2023-06-27T12:40:20.404 INFO:tasks.rados.rados.0.smithi138.stdout:552: left oid 35 (ObjNum 271 snap 70 seq_num 271) 2023-06-27T12:40:20.404 INFO:tasks.rados.rados.0.smithi138.stdout:559: expect (ObjNum 233 snap 58 seq_num 233) 2023-06-27T12:40:20.732 INFO:tasks.rados.rados.0.smithi138.stdout:560: finishing write tid 1 to smithi138138488-4 2023-06-27T12:40:20.732 INFO:tasks.rados.rados.0.smithi138.stdout:560: finishing write tid 2 to smithi138138488-4 2023-06-27T12:40:20.732 INFO:tasks.rados.rados.0.smithi138.stdout:560: finishing write tid 3 to smithi138138488-4 2023-06-27T12:40:20.732 INFO:tasks.rados.rados.0.smithi138.stdout:560: finishing write tid 4 to smithi138138488-4 2023-06-27T12:40:20.732 INFO:tasks.rados.rados.0.smithi138.stdout:560: oid 4 updating version 0 to 73 2023-06-27T12:40:20.733 INFO:tasks.rados.rados.0.smithi138.stdout:560: oid 4 version 73 is already newer than 73 2023-06-27T12:40:20.733 INFO:tasks.rados.rados.0.smithi138.stdout:560: oid 4 version 73 is already newer than 73 2023-06-27T12:40:20.733 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 73 (ObjNum 273 snap 70 seq_num 273) dirty exists 2023-06-27T12:40:20.733 INFO:tasks.rados.rados.0.smithi138.stdout:560: left oid 4 (ObjNum 273 snap 70 seq_num 273) 2023-06-27T12:40:21.398 INFO:tasks.rados.rados.0.smithi138.stdout:549: done (8 left) 2023-06-27T12:40:21.399 INFO:tasks.rados.rados.0.smithi138.stdout:550: done (7 left) 2023-06-27T12:40:21.399 INFO:tasks.rados.rados.0.smithi138.stdout:552: done (6 left) 2023-06-27T12:40:21.399 INFO:tasks.rados.rados.0.smithi138.stdout:553: done (5 left) 2023-06-27T12:40:21.399 INFO:tasks.rados.rados.0.smithi138.stdout:554: done (4 left) 2023-06-27T12:40:21.399 INFO:tasks.rados.rados.0.smithi138.stdout:557: done (3 left) 2023-06-27T12:40:21.400 INFO:tasks.rados.rados.0.smithi138.stdout:559: done (2 left) 2023-06-27T12:40:21.400 INFO:tasks.rados.rados.0.smithi138.stdout:560: done (1 left) 2023-06-27T12:40:21.400 INFO:tasks.rados.rados.0.smithi138.stdout:562: done (0 left) 2023-06-27T12:40:21.400 INFO:tasks.rados.rados.0.smithi138.stdout:563: write oid 6 current snap is 71 2023-06-27T12:40:21.400 INFO:tasks.rados.rados.0.smithi138.stdout:563: seq_num 274 ranges {605118=406934,1617125=706417} 2023-06-27T12:40:21.403 INFO:tasks.rados.rados.0.smithi138.stdout:563: writing smithi138138488-6 from 605118 to 1012052 tid 1 2023-06-27T12:40:21.411 INFO:tasks.rados.rados.0.smithi138.stdout:563: writing smithi138138488-6 from 1617125 to 2323542 tid 2 2023-06-27T12:40:21.411 INFO:tasks.rados.rados.0.smithi138.stdout:564: snap_create 2023-06-27T12:40:21.480 INFO:tasks.rados.rados.0.smithi138.stdout:563: finishing write tid 1 to smithi138138488-6 2023-06-27T12:40:21.482 INFO:tasks.rados.rados.0.smithi138.stdout:563: finishing write tid 2 to smithi138138488-6 2023-06-27T12:40:21.486 INFO:tasks.rados.rados.0.smithi138.stdout:563: finishing write tid 3 to smithi138138488-6 2023-06-27T12:40:21.487 INFO:tasks.rados.rados.0.smithi138.stdout:563: finishing write tid 4 to smithi138138488-6 2023-06-27T12:40:21.487 INFO:tasks.rados.rados.0.smithi138.stdout:563: oid 6 updating version 0 to 69 2023-06-27T12:40:21.488 INFO:tasks.rados.rados.0.smithi138.stdout:563: oid 6 version 69 is already newer than 69 2023-06-27T12:40:21.488 INFO:tasks.rados.rados.0.smithi138.stdout:563: oid 6 version 69 is already newer than 69 2023-06-27T12:40:21.488 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 69 (ObjNum 274 snap 71 seq_num 274) dirty exists 2023-06-27T12:40:21.488 INFO:tasks.rados.rados.0.smithi138.stdout:563: left oid 6 (ObjNum 274 snap 71 seq_num 274) 2023-06-27T12:40:22.088 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:22.088 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:22.089 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:22.217 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:22.210+0000 7f267f812700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:22.220 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:22.217+0000 7f267f812700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:22.289 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:40:22.399 INFO:tasks.rados.rados.0.smithi138.stdout:563: done (1 left) 2023-06-27T12:40:22.399 INFO:tasks.rados.rados.0.smithi138.stdout:564: done (0 left) 2023-06-27T12:40:22.400 INFO:tasks.rados.rados.0.smithi138.stdout:565: write oid 22 current snap is 72 2023-06-27T12:40:22.400 INFO:tasks.rados.rados.0.smithi138.stdout:565: seq_num 275 ranges {621925=482183,1559254=566428} 2023-06-27T12:40:22.405 INFO:tasks.rados.rados.0.smithi138.stdout:565: writing smithi138138488-22 from 621925 to 1104108 tid 1 2023-06-27T12:40:22.411 INFO:tasks.rados.rados.0.smithi138.stdout:565: writing smithi138138488-22 from 1559254 to 2125682 tid 2 2023-06-27T12:40:22.411 INFO:tasks.rados.rados.0.smithi138.stdout:566: snap_remove snap 42 2023-06-27T12:40:22.419 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:22.414+0000 7f500c129700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:22.422 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:22.419+0000 7f500c129700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:22.430 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:22.432 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:22.446 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2023-06-27T12:40:22.394658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98054689,"num_objects":191,"num_object_clones":133,"num_object_copies":573,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":942,"num_read_kb":598948,"num_write":1062,"num_write_kb":269667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1427,"ondisk_log_size":1427,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":671424,"kb_used_data":546716,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376815936,"statfs":{"total":386547056640,"available":385859518464,"internally_reserved":0,"allocated":559837184,"data_stored":651016052,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-789632,"num_objects":11,"num_object_clones":9,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":31,"num_read_kb":17077,"num_write":48,"num_write_kb":13273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.936767"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":287,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.472683+0000","last_change":"2023-06-27T12:40:18.479159+0000","last_active":"2023-06-27T12:40:18.472683+0000","last_peered":"2023-06-27T12:40:18.472683+0000","last_clean":"2023-06-27T12:40:18.472683+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:18.472683+0000","last_undegraded":"2023-06-27T12:40:18.472683+0000","last_fullsized":"2023-06-27T12:40:18.472683+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":401,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.592247+0000","last_change":"2023-06-27T12:40:17.592470+0000","last_active":"2023-06-27T12:40:17.592247+0000","last_peered":"2023-06-27T12:40:17.592247+0000","last_clean":"2023-06-27T12:40:17.592247+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:17.592247+0000","last_undegraded":"2023-06-27T12:40:17.592247+0000","last_fullsized":"2023-06-27T12:40:17.592247+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":407,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.565139+0000","last_change":"2023-06-27T12:40:17.566217+0000","last_active":"2023-06-27T12:40:17.565139+0000","last_peered":"2023-06-27T12:40:17.565139+0000","last_clean":"2023-06-27T12:40:17.565139+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:17.565139+0000","last_undegraded":"2023-06-27T12:40:17.565139+0000","last_fullsized":"2023-06-27T12:40:17.565139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":60,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"141'57","reported_seq":362,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.503775+0000","last_change":"2023-06-27T12:40:18.504363+0000","last_active":"2023-06-27T12:40:18.503775+0000","last_peered":"2023-06-27T12:40:18.503775+0000","last_clean":"2023-06-27T12:40:18.503775+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:18.503775+0000","last_undegraded":"2023-06-27T12:40:18.503775+0000","last_fullsized":"2023-06-27T12:40:18.503775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563150,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":55,"num_read_kb":27198,"num_write":47,"num_write_kb":12924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":329,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.472589+0000","last_change":"2023-06-27T12:40:18.475142+0000","last_active":"2023-06-27T12:40:18.472589+0000","last_peered":"2023-06-27T12:40:18.472589+0000","last_clean":"2023-06-27T12:40:18.472589+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:18.472589+0000","last_undegraded":"2023-06-27T12:40:18.472589+0000","last_fullsized":"2023-06-27T12:40:18.472589+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":454,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.481048+0000","last_change":"2023-06-27T12:40:18.481171+0000","last_active":"2023-06-27T12:40:18.481048+0000","last_peered":"2023-06-27T12:40:18.481048+0000","last_clean":"2023-06-27T12:40:18.481048+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:18.481048+0000","last_undegraded":"2023-06-27T12:40:18.481048+0000","last_fullsized":"2023-06-27T12:40:18.481048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":154,"num_read_kb":103228,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"147'87","reported_seq":366,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.472585+0000","last_change":"2023-06-27T12:40:18.507769+0000","last_active":"2023-06-27T12:40:18.472585+0000","last_peered":"2023-06-27T12:40:18.472585+0000","last_clean":"2023-06-27T12:40:18.472585+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:18.472585+0000","last_undegraded":"2023-06-27T12:40:18.472585+0000","last_fullsized":"2023-06-27T12:40:18.472585+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5546343,"num_objects":13,"num_object_clones":10,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":55,"num_read_kb":30117,"num_write":59,"num_write_kb":17358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"117'32","reported_seq":327,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.504857+0000","last_change":"2023-06-27T12:40:17.504989+0000","last_active":"2023-06-27T12:40:17.504857+0000","last_peered":"2023-06-27T12:40:17.504857+0000","last_clean":"2023-06-27T12:40:17.504857+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:17.504857+0000","last_undegraded":"2023-06-27T12:40:17.504857+0000","last_fullsized":"2023-06-27T12:40:17.504857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3798382,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25160,"num_write":23,"num_write_kb":7301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":233,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.502962+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:17.502962+0000","last_peered":"2023-06-27T12:40:17.502962+0000","last_clean":"2023-06-27T12:40:17.502962+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:17.502962+0000","last_undegraded":"2023-06-27T12:40:17.502962+0000","last_fullsized":"2023-06-27T12:40:17.502962+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":285,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.493223+0000","last_change":"2023-06-27T12:40:17.503226+0000","last_active":"2023-06-27T12:40:17.493223+0000","last_peered":"2023-06-27T12:40:17.493223+0000","last_clean":"2023-06-27T12:40:17.493223+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:17.493223+0000","last_undegraded":"2023-06-27T12:40:17.493223+0000","last_fullsized":"2023-06-27T12:40:17.493223+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":225,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.474451+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:18.474451+0000","last_peered":"2023-06-27T12:40:18.474451+0000","last_clean":"2023-06-27T12:40:18.474451+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:18.474451+0000","last_undegraded":"2023-06-27T12:40:18.474451+0000","last_fullsized":"2023-06-27T12:40:18.474451+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"146'55","reported_seq":351,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.500281+0000","last_change":"2023-06-27T12:40:17.593035+0000","last_active":"2023-06-27T12:40:17.500281+0000","last_peered":"2023-06-27T12:40:17.500281+0000","last_clean":"2023-06-27T12:40:17.500281+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:17.500281+0000","last_undegraded":"2023-06-27T12:40:17.500281+0000","last_fullsized":"2023-06-27T12:40:17.500281+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5798448,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":34,"num_read_kb":31322,"num_write":42,"num_write_kb":11438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":229,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.487968+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:17.487968+0000","last_peered":"2023-06-27T12:40:17.487968+0000","last_clean":"2023-06-27T12:40:17.487968+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:17.487968+0000","last_undegraded":"2023-06-27T12:40:17.487968+0000","last_fullsized":"2023-06-27T12:40:17.487968+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"149'216","reported_seq":460,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.467620+0000","last_change":"2023-06-27T12:40:18.493592+0000","last_active":"2023-06-27T12:40:18.467620+0000","last_peered":"2023-06-27T12:40:18.467620+0000","last_clean":"2023-06-27T12:40:18.467620+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:18.467620+0000","last_undegraded":"2023-06-27T12:40:18.467620+0000","last_fullsized":"2023-06-27T12:40:18.467620+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7630444,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":86,"num_read_kb":71886,"num_write":128,"num_write_kb":38201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":229,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.493983+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:17.493983+0000","last_peered":"2023-06-27T12:40:17.493983+0000","last_clean":"2023-06-27T12:40:17.493983+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:17.493983+0000","last_undegraded":"2023-06-27T12:40:17.493983+0000","last_fullsized":"2023-06-27T12:40:17.493983+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":358,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.466892+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:18.466892+0000","last_peered":"2023-06-27T12:40:18.466892+0000","last_clean":"2023-06-27T12:40:18.466892+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:18.466892+0000","last_undegraded":"2023-06-27T12:40:18.466892+0000","last_fullsized":"2023-06-27T12:40:18.466892+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"148'153","reported_seq":458,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.556989+0000","last_change":"2023-06-27T12:40:17.557245+0000","last_active":"2023-06-27T12:40:17.556989+0000","last_peered":"2023-06-27T12:40:17.556989+0000","last_clean":"2023-06-27T12:40:17.556989+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:17.556989+0000","last_undegraded":"2023-06-27T12:40:17.556989+0000","last_fullsized":"2023-06-27T12:40:17.556989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15596929,"num_objects":22,"num_object_clones":16,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":131,"num_read_kb":64219,"num_write":118,"num_write_kb":34078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":225,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.469967+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:18.469967+0000","last_peered":"2023-06-27T12:40:18.469967+0000","last_clean":"2023-06-27T12:40:18.469967+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:18.469967+0000","last_undegraded":"2023-06-27T12:40:18.469967+0000","last_fullsized":"2023-06-27T12:40:18.469967+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"146'69","reported_seq":352,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.470483+0000","last_change":"2023-06-27T12:40:18.474109+0000","last_active":"2023-06-27T12:40:18.470483+0000","last_peered":"2023-06-27T12:40:18.470483+0000","last_clean":"2023-06-27T12:40:18.470483+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:18.470483+0000","last_undegraded":"2023-06-27T12:40:18.470483+0000","last_fullsized":"2023-06-27T12:40:18.470483+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6252237,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":32,"num_read_kb":24674,"num_write":49,"num_write_kb":14223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":227,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.468785+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:18.468785+0000","last_peered":"2023-06-27T12:40:18.468785+0000","last_clean":"2023-06-27T12:40:18.468785+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:18.468785+0000","last_undegraded":"2023-06-27T12:40:18.468785+0000","last_fullsized":"2023-06-27T12:40:18.468785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":381,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.489565+0000","last_change":"2023-06-27T12:40:17.591493+0000","last_active":"2023-06-27T12:40:17.489565+0000","last_peered":"2023-06-27T12:40:17.489565+0000","last_clean":"2023-06-27T12:40:17.489565+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:17.489565+0000","last_undegraded":"2023-06-27T12:40:17.489565+0000","last_fullsized":"2023-06-27T12:40:17.489565+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":335,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-06-27T12:40:18.471486+0000","last_change":"2023-06-27T12:40:18.474628+0000","last_active":"2023-06-27T12:40:18.471486+0000","last_peered":"2023-06-27T12:40:18.471486+0000","last_clean":"2023-06-27T12:40:18.471486+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:18.471486+0000","last_undegraded":"2023-06-27T12:40:18.471486+0000","last_fullsized":"2023-06-27T12:40:18.471486+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":233,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.488671+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:17.488671+0000","last_peered":"2023-06-27T12:40:17.488671+0000","last_clean":"2023-06-27T12:40:17.488671+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:17.488671+0000","last_undegraded":"2023-06-27T12:40:17.488671+0000","last_fullsized":"2023-06-27T12:40:17.488671+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":229,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.487097+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:17.487097+0000","last_peered":"2023-06-27T12:40:17.487097+0000","last_clean":"2023-06-27T12:40:17.487097+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:17.487097+0000","last_undegraded":"2023-06-27T12:40:17.487097+0000","last_fullsized":"2023-06-27T12:40:17.487097+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"144'86","reported_seq":371,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-06-27T12:40:17.498147+0000","last_change":"2023-06-27T12:40:17.639962+0000","last_active":"2023-06-27T12:40:17.498147+0000","last_peered":"2023-06-27T12:40:17.498147+0000","last_clean":"2023-06-27T12:40:17.498147+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:17.498147+0000","last_undegraded":"2023-06-27T12:40:17.498147+0000","last_fullsized":"2023-06-27T12:40:17.498147+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4353079,"num_objects":11,"num_object_clones":8,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":42,"num_read_kb":42937,"num_write":53,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":97464321,"num_objects":189,"num_object_clones":133,"num_object_copies":567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":839,"num_read_kb":598857,"num_write":925,"num_write_kb":267773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1330,"ondisk_log_size":1330,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673105,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":176556,"kb_used_data":145636,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94195284,"statfs":{"total":96636764160,"available":96455970816,"internally_reserved":0,"allocated":149131264,"data_stored":172425353,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673105,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":154588,"kb_used_data":123668,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94217252,"statfs":{"total":96636764160,"available":96478466048,"internally_reserved":0,"allocated":126636032,"data_stored":147545819,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705810,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":162052,"kb_used_data":131132,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94209788,"statfs":{"total":96636764160,"available":96470822912,"internally_reserved":0,"allocated":134279168,"data_stored":154601584,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673105,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":178228,"kb_used_data":146280,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94193612,"statfs":{"total":96636764160,"available":96454258688,"internally_reserved":0,"allocated":149790720,"data_stored":176443296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:23.403 INFO:tasks.rados.rados.0.smithi138.stdout:566: done (1 left) 2023-06-27T12:40:23.403 INFO:tasks.rados.rados.0.smithi138.stdout:567: snap_create 2023-06-27T12:40:23.403 INFO:tasks.rados.rados.0.smithi138.stdout:565: finishing write tid 1 to smithi138138488-22 2023-06-27T12:40:23.404 INFO:tasks.rados.rados.0.smithi138.stdout:565: finishing write tid 2 to smithi138138488-22 2023-06-27T12:40:23.404 INFO:tasks.rados.rados.0.smithi138.stdout:565: finishing write tid 3 to smithi138138488-22 2023-06-27T12:40:23.404 INFO:tasks.rados.rados.0.smithi138.stdout:565: finishing write tid 4 to smithi138138488-22 2023-06-27T12:40:23.404 INFO:tasks.rados.rados.0.smithi138.stdout:565: oid 22 updating version 0 to 120 2023-06-27T12:40:23.404 INFO:tasks.rados.rados.0.smithi138.stdout:565: oid 22 version 120 is already newer than 120 2023-06-27T12:40:23.404 INFO:tasks.rados.rados.0.smithi138.stdout:565: oid 22 version 120 is already newer than 120 2023-06-27T12:40:23.404 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 120 (ObjNum 275 snap 72 seq_num 275) dirty exists 2023-06-27T12:40:23.405 INFO:tasks.rados.rados.0.smithi138.stdout:565: left oid 22 (ObjNum 275 snap 72 seq_num 275) 2023-06-27T12:40:24.406 INFO:tasks.rados.rados.0.smithi138.stdout:565: done (1 left) 2023-06-27T12:40:24.406 INFO:tasks.rados.rados.0.smithi138.stdout:567: done (0 left) 2023-06-27T12:40:24.406 INFO:tasks.rados.rados.0.smithi138.stdout:568: snap_create 2023-06-27T12:40:24.520 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:40:24.650 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:24.645+0000 7f0ceba0b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:24.653 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:24.650+0000 7f0ceba0b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:25.410 INFO:tasks.rados.rados.0.smithi138.stdout:568: done (0 left) 2023-06-27T12:40:25.411 INFO:tasks.rados.rados.0.smithi138.stdout:569: snap_remove snap 69 2023-06-27T12:40:26.410 INFO:tasks.rados.rados.0.smithi138.stdout:569: done (0 left) 2023-06-27T12:40:26.411 INFO:tasks.rados.rados.0.smithi138.stdout:570: write oid 46 current snap is 74 2023-06-27T12:40:26.411 INFO:tasks.rados.rados.0.smithi138.stdout:570: seq_num 276 ranges {638732=557432,1617736=710086} 2023-06-27T12:40:26.417 INFO:tasks.rados.rados.0.smithi138.stdout:570: writing smithi138138488-46 from 638732 to 1196164 tid 1 2023-06-27T12:40:26.423 INFO:tasks.rados.rados.0.smithi138.stdout:570: writing smithi138138488-46 from 1617736 to 2327822 tid 2 2023-06-27T12:40:26.424 INFO:tasks.rados.rados.0.smithi138.stdout:571: snap_remove snap 50 2023-06-27T12:40:26.725 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:40:26.866 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:26.860+0000 7f1727e93700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:26.869 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:26.865+0000 7f1727e93700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:27.424 INFO:tasks.rados.rados.0.smithi138.stdout:571: done (1 left) 2023-06-27T12:40:27.425 INFO:tasks.rados.rados.0.smithi138.stdout:572: delete oid 37 current snap is 74 2023-06-27T12:40:27.425 INFO:tasks.rados.rados.0.smithi138.stdout:570: finishing write tid 1 to smithi138138488-46 2023-06-27T12:40:27.425 INFO:tasks.rados.rados.0.smithi138.stdout:570: finishing write tid 2 to smithi138138488-46 2023-06-27T12:40:27.425 INFO:tasks.rados.rados.0.smithi138.stdout:570: finishing write tid 3 to smithi138138488-46 2023-06-27T12:40:27.425 INFO:tasks.rados.rados.0.smithi138.stdout:570: finishing write tid 4 to smithi138138488-46 2023-06-27T12:40:27.425 INFO:tasks.rados.rados.0.smithi138.stdout:570: oid 46 updating version 0 to 76 2023-06-27T12:40:27.425 INFO:tasks.rados.rados.0.smithi138.stdout:570: oid 46 version 76 is already newer than 76 2023-06-27T12:40:27.426 INFO:tasks.rados.rados.0.smithi138.stdout:570: oid 46 version 76 is already newer than 76 2023-06-27T12:40:27.426 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 76 (ObjNum 276 snap 74 seq_num 276) dirty exists 2023-06-27T12:40:27.426 INFO:tasks.rados.rados.0.smithi138.stdout:570: left oid 46 (ObjNum 276 snap 74 seq_num 276) 2023-06-27T12:40:27.448 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:27.449 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:27.449 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:27.531 INFO:tasks.rados.rados.0.smithi138.stdout:570: done (1 left) 2023-06-27T12:40:27.531 INFO:tasks.rados.rados.0.smithi138.stdout:572: done (0 left) 2023-06-27T12:40:27.531 INFO:tasks.rados.rados.0.smithi138.stdout:573: delete oid 48 current snap is 74 2023-06-27T12:40:27.566 INFO:tasks.rados.rados.0.smithi138.stdout:573: done (0 left) 2023-06-27T12:40:27.566 INFO:tasks.rados.rados.0.smithi138.stdout:574: read oid 9 snap -1 2023-06-27T12:40:27.567 INFO:tasks.rados.rados.0.smithi138.stdout:574: expect (ObjNum 270 snap 70 seq_num 270) 2023-06-27T12:40:27.567 INFO:tasks.rados.rados.0.smithi138.stdout:575: snap_create 2023-06-27T12:40:27.580 INFO:tasks.rados.rados.0.smithi138.stdout:574: expect (ObjNum 270 snap 70 seq_num 270) 2023-06-27T12:40:27.602 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:27.594+0000 7fb146dd6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:27.604 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:27.601+0000 7fb146dd6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:27.813 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:27.815 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:27.827 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2023-06-27T12:40:27.418972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":101281666,"num_objects":193,"num_object_clones":135,"num_object_copies":579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":21,"num_read":952,"num_read_kb":609535,"num_write":1074,"num_write_kb":272931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1479,"ondisk_log_size":1479,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":674192,"kb_used_data":550512,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376813168,"statfs":{"total":386547056640,"available":385856684032,"internally_reserved":0,"allocated":563724288,"data_stored":653735106,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3226977,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":10,"num_read_kb":10587,"num_write":12,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020047"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":297,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.810638+0000","last_change":"2023-06-27T12:40:20.567089+0000","last_active":"2023-06-27T12:40:22.810638+0000","last_peered":"2023-06-27T12:40:22.810638+0000","last_clean":"2023-06-27T12:40:22.810638+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:22.810638+0000","last_undegraded":"2023-06-27T12:40:22.810638+0000","last_fullsized":"2023-06-27T12:40:22.810638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":417,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-06-27T12:40:23.440438+0000","last_change":"2023-06-27T12:40:23.443473+0000","last_active":"2023-06-27T12:40:23.440438+0000","last_peered":"2023-06-27T12:40:23.440438+0000","last_clean":"2023-06-27T12:40:23.440438+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:23.440438+0000","last_undegraded":"2023-06-27T12:40:23.440438+0000","last_fullsized":"2023-06-27T12:40:23.440438+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":417,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.835205+0000","last_change":"2023-06-27T12:40:22.835205+0000","last_active":"2023-06-27T12:40:22.835205+0000","last_peered":"2023-06-27T12:40:22.835205+0000","last_clean":"2023-06-27T12:40:22.835205+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:22.835205+0000","last_undegraded":"2023-06-27T12:40:22.835205+0000","last_fullsized":"2023-06-27T12:40:22.835205+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":60,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"141'57","reported_seq":373,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.814211+0000","last_change":"2023-06-27T12:40:20.586030+0000","last_active":"2023-06-27T12:40:22.814211+0000","last_peered":"2023-06-27T12:40:22.814211+0000","last_clean":"2023-06-27T12:40:22.814211+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:22.814211+0000","last_undegraded":"2023-06-27T12:40:22.814211+0000","last_fullsized":"2023-06-27T12:40:22.814211+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563150,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":55,"num_read_kb":27198,"num_write":47,"num_write_kb":12924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":339,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.408374+0000","last_change":"2023-06-27T12:40:20.564923+0000","last_active":"2023-06-27T12:40:22.408374+0000","last_peered":"2023-06-27T12:40:22.408374+0000","last_clean":"2023-06-27T12:40:22.408374+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:22.408374+0000","last_undegraded":"2023-06-27T12:40:22.408374+0000","last_fullsized":"2023-06-27T12:40:22.408374+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":469,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.817219+0000","last_change":"2023-06-27T12:40:20.570718+0000","last_active":"2023-06-27T12:40:22.817219+0000","last_peered":"2023-06-27T12:40:22.817219+0000","last_clean":"2023-06-27T12:40:22.817219+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:22.817219+0000","last_undegraded":"2023-06-27T12:40:22.817219+0000","last_fullsized":"2023-06-27T12:40:22.817219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":113812,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"147'87","reported_seq":376,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.808556+0000","last_change":"2023-06-27T12:40:20.601907+0000","last_active":"2023-06-27T12:40:22.808556+0000","last_peered":"2023-06-27T12:40:22.808556+0000","last_clean":"2023-06-27T12:40:22.808556+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:22.808556+0000","last_undegraded":"2023-06-27T12:40:22.808556+0000","last_fullsized":"2023-06-27T12:40:22.808556+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5546343,"num_objects":13,"num_object_clones":10,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":55,"num_read_kb":30117,"num_write":59,"num_write_kb":17358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"117'32","reported_seq":337,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.815619+0000","last_change":"2023-06-27T12:40:21.924049+0000","last_active":"2023-06-27T12:40:22.815619+0000","last_peered":"2023-06-27T12:40:22.815619+0000","last_clean":"2023-06-27T12:40:22.815619+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:22.815619+0000","last_undegraded":"2023-06-27T12:40:22.815619+0000","last_fullsized":"2023-06-27T12:40:22.815619+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3798382,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25160,"num_write":23,"num_write_kb":7301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":243,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-06-27T12:40:23.414191+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:23.414191+0000","last_peered":"2023-06-27T12:40:23.414191+0000","last_clean":"2023-06-27T12:40:23.414191+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:23.414191+0000","last_undegraded":"2023-06-27T12:40:23.414191+0000","last_fullsized":"2023-06-27T12:40:23.414191+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":294,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.815934+0000","last_change":"2023-06-27T12:40:20.442975+0000","last_active":"2023-06-27T12:40:22.815934+0000","last_peered":"2023-06-27T12:40:22.815934+0000","last_clean":"2023-06-27T12:40:22.815934+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:22.815934+0000","last_undegraded":"2023-06-27T12:40:22.815934+0000","last_fullsized":"2023-06-27T12:40:22.815934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":233,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.807978+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:22.807978+0000","last_peered":"2023-06-27T12:40:22.807978+0000","last_clean":"2023-06-27T12:40:22.807978+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:22.807978+0000","last_undegraded":"2023-06-27T12:40:22.807978+0000","last_fullsized":"2023-06-27T12:40:22.807978+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"153'69","reported_seq":369,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-06-27T12:40:23.408710+0000","last_change":"2023-06-27T12:40:23.463494+0000","last_active":"2023-06-27T12:40:23.408710+0000","last_peered":"2023-06-27T12:40:23.408710+0000","last_clean":"2023-06-27T12:40:23.408710+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:23.408710+0000","last_undegraded":"2023-06-27T12:40:23.408710+0000","last_fullsized":"2023-06-27T12:40:23.408710+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050553,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":35,"num_read_kb":31323,"num_write":46,"num_write_kb":12526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":235,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.827735+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:22.827735+0000","last_peered":"2023-06-27T12:40:22.827735+0000","last_clean":"2023-06-27T12:40:22.827735+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:22.827735+0000","last_undegraded":"2023-06-27T12:40:22.827735+0000","last_fullsized":"2023-06-27T12:40:22.827735+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"149'216","reported_seq":471,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.406615+0000","last_change":"2023-06-27T12:40:20.562641+0000","last_active":"2023-06-27T12:40:22.406615+0000","last_peered":"2023-06-27T12:40:22.406615+0000","last_clean":"2023-06-27T12:40:22.406615+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:22.406615+0000","last_undegraded":"2023-06-27T12:40:22.406615+0000","last_fullsized":"2023-06-27T12:40:22.406615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7630444,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":86,"num_read_kb":71886,"num_write":128,"num_write_kb":38201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":235,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.823950+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:22.823950+0000","last_peered":"2023-06-27T12:40:22.823950+0000","last_clean":"2023-06-27T12:40:22.823950+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:22.823950+0000","last_undegraded":"2023-06-27T12:40:22.823950+0000","last_fullsized":"2023-06-27T12:40:22.823950+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":366,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.409634+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:22.409634+0000","last_peered":"2023-06-27T12:40:22.409634+0000","last_clean":"2023-06-27T12:40:22.409634+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:22.409634+0000","last_undegraded":"2023-06-27T12:40:22.409634+0000","last_fullsized":"2023-06-27T12:40:22.409634+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"148'153","reported_seq":467,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.822568+0000","last_change":"2023-06-27T12:40:21.956974+0000","last_active":"2023-06-27T12:40:22.822568+0000","last_peered":"2023-06-27T12:40:22.822568+0000","last_clean":"2023-06-27T12:40:22.822568+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:22.822568+0000","last_undegraded":"2023-06-27T12:40:22.822568+0000","last_fullsized":"2023-06-27T12:40:22.822568+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15596929,"num_objects":22,"num_object_clones":16,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":131,"num_read_kb":64219,"num_write":118,"num_write_kb":34078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":233,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.810463+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:22.810463+0000","last_peered":"2023-06-27T12:40:22.810463+0000","last_clean":"2023-06-27T12:40:22.810463+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:22.810463+0000","last_undegraded":"2023-06-27T12:40:22.810463+0000","last_fullsized":"2023-06-27T12:40:22.810463+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"151'73","reported_seq":366,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.410336+0000","last_change":"2023-06-27T12:40:20.568884+0000","last_active":"2023-06-27T12:40:22.410336+0000","last_peered":"2023-06-27T12:40:22.410336+0000","last_clean":"2023-06-27T12:40:22.410336+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:22.410336+0000","last_undegraded":"2023-06-27T12:40:22.410336+0000","last_fullsized":"2023-06-27T12:40:22.410336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840548,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":33,"num_read_kb":24675,"num_write":53,"num_write_kb":15374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":235,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.404749+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:22.404749+0000","last_peered":"2023-06-27T12:40:22.404749+0000","last_clean":"2023-06-27T12:40:22.404749+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:22.404749+0000","last_undegraded":"2023-06-27T12:40:22.404749+0000","last_fullsized":"2023-06-27T12:40:22.404749+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":395,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-06-27T12:40:23.409972+0000","last_change":"2023-06-27T12:40:23.461777+0000","last_active":"2023-06-27T12:40:23.409972+0000","last_peered":"2023-06-27T12:40:23.409972+0000","last_clean":"2023-06-27T12:40:23.409972+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:23.409972+0000","last_undegraded":"2023-06-27T12:40:23.409972+0000","last_fullsized":"2023-06-27T12:40:23.409972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":345,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.414769+0000","last_change":"2023-06-27T12:40:20.569109+0000","last_active":"2023-06-27T12:40:22.414769+0000","last_peered":"2023-06-27T12:40:22.414769+0000","last_clean":"2023-06-27T12:40:22.414769+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:22.414769+0000","last_undegraded":"2023-06-27T12:40:22.414769+0000","last_fullsized":"2023-06-27T12:40:22.414769+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":243,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-06-27T12:40:23.411389+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:23.411389+0000","last_peered":"2023-06-27T12:40:23.411389+0000","last_clean":"2023-06-27T12:40:23.411389+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:23.411389+0000","last_undegraded":"2023-06-27T12:40:23.411389+0000","last_fullsized":"2023-06-27T12:40:23.411389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":235,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-06-27T12:40:22.823796+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:22.823796+0000","last_peered":"2023-06-27T12:40:22.823796+0000","last_clean":"2023-06-27T12:40:22.823796+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:22.823796+0000","last_undegraded":"2023-06-27T12:40:22.823796+0000","last_fullsized":"2023-06-27T12:40:22.823796+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":392,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-06-27T12:40:23.462459+0000","last_change":"2023-06-27T12:40:23.462586+0000","last_active":"2023-06-27T12:40:23.462459+0000","last_peered":"2023-06-27T12:40:23.462459+0000","last_clean":"2023-06-27T12:40:23.462459+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:23.462459+0000","last_undegraded":"2023-06-27T12:40:23.462459+0000","last_fullsized":"2023-06-27T12:40:23.462459+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":43,"num_read_kb":42938,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":100691298,"num_objects":191,"num_object_clones":135,"num_object_copies":573,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":21,"num_read":849,"num_read_kb":609444,"num_write":937,"num_write_kb":271037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1382,"ondisk_log_size":1382,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673110,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":178788,"kb_used_data":147868,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94193052,"statfs":{"total":96636764160,"available":96453685248,"internally_reserved":0,"allocated":151416832,"data_stored":174509920,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673110,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":154692,"kb_used_data":123772,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94217148,"statfs":{"total":96636764160,"available":96478359552,"internally_reserved":0,"allocated":126742528,"data_stored":147272895,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163352,"kb_used_data":132432,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94208488,"statfs":{"total":96636764160,"available":96469491712,"internally_reserved":0,"allocated":135610368,"data_stored":156265794,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673110,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":177360,"kb_used_data":146440,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94194480,"statfs":{"total":96636764160,"available":96455147520,"internally_reserved":0,"allocated":149954560,"data_stored":175686497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:28.429 INFO:tasks.rados.rados.0.smithi138.stdout:574: done (1 left) 2023-06-27T12:40:28.429 INFO:tasks.rados.rados.0.smithi138.stdout:575: done (0 left) 2023-06-27T12:40:28.429 INFO:tasks.rados.rados.0.smithi138.stdout:576: write oid 45 current snap is 75 2023-06-27T12:40:28.429 INFO:tasks.rados.rados.0.smithi138.stdout:576: seq_num 279 ranges {689153=783179,2076829=457413} 2023-06-27T12:40:28.437 INFO:tasks.rados.rados.0.smithi138.stdout:576: writing smithi138138488-45 from 689153 to 1472332 tid 1 2023-06-27T12:40:28.444 INFO:tasks.rados.rados.0.smithi138.stdout:576: writing smithi138138488-45 from 2076829 to 2534242 tid 2 2023-06-27T12:40:28.445 INFO:tasks.rados.rados.0.smithi138.stdout:577: read oid 2 snap 58 2023-06-27T12:40:28.445 INFO:tasks.rados.rados.0.smithi138.stdout:577: expect deleted 2023-06-27T12:40:28.445 INFO:tasks.rados.rados.0.smithi138.stdout:578: snap_remove snap 70 2023-06-27T12:40:28.938 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:40:29.066 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:29.060+0000 7f037b0fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:29.068 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:29.065+0000 7f037b0fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:29.432 INFO:tasks.rados.rados.0.smithi138.stdout:578: done (2 left) 2023-06-27T12:40:29.433 INFO:tasks.rados.rados.0.smithi138.stdout:579: snap_remove snap 74 2023-06-27T12:40:30.400 INFO:tasks.rados.rados.0.smithi138.stdout:579: done (2 left) 2023-06-27T12:40:30.401 INFO:tasks.rados.rados.0.smithi138.stdout:580: write (excl) oid 40 current snap is 75 2023-06-27T12:40:30.401 INFO:tasks.rados.rados.0.smithi138.stdout:580: seq_num 280 ranges {705960=458428,1735311=717424} 2023-06-27T12:40:30.407 INFO:tasks.rados.rados.0.smithi138.stdout:580: writing smithi138138488-40 from 705960 to 1164388 tid 1 2023-06-27T12:40:30.414 INFO:tasks.rados.rados.0.smithi138.stdout:580: writing smithi138138488-40 from 1735311 to 2452735 tid 2 2023-06-27T12:40:30.415 INFO:tasks.rados.rados.0.smithi138.stdout:581: read oid 50 snap -1 2023-06-27T12:40:30.415 INFO:tasks.rados.rados.0.smithi138.stdout:581: expect (ObjNum 226 snap 55 seq_num 226) 2023-06-27T12:40:30.415 INFO:tasks.rados.rados.0.smithi138.stdout:582: write oid 19 current snap is 75 2023-06-27T12:40:30.415 INFO:tasks.rados.rados.0.smithi138.stdout:576: finishing write tid 1 to smithi138138488-45 2023-06-27T12:40:30.415 INFO:tasks.rados.rados.0.smithi138.stdout:582: seq_num 281 ranges {722767=533677,1793793=461082} 2023-06-27T12:40:30.422 INFO:tasks.rados.rados.0.smithi138.stdout:582: writing smithi138138488-19 from 722767 to 1256444 tid 1 2023-06-27T12:40:30.428 INFO:tasks.rados.rados.0.smithi138.stdout:582: writing smithi138138488-19 from 1793793 to 2254875 tid 2 2023-06-27T12:40:30.428 INFO:tasks.rados.rados.0.smithi138.stdout:583: snap_create 2023-06-27T12:40:30.428 INFO:tasks.rados.rados.0.smithi138.stdout:576: finishing write tid 2 to smithi138138488-45 2023-06-27T12:40:30.428 INFO:tasks.rados.rados.0.smithi138.stdout:576: finishing write tid 3 to smithi138138488-45 2023-06-27T12:40:30.428 INFO:tasks.rados.rados.0.smithi138.stdout:576: finishing write tid 4 to smithi138138488-45 2023-06-27T12:40:30.428 INFO:tasks.rados.rados.0.smithi138.stdout:576: oid 45 updating version 0 to 233 2023-06-27T12:40:30.429 INFO:tasks.rados.rados.0.smithi138.stdout:576: oid 45 version 233 is already newer than 233 2023-06-27T12:40:30.429 INFO:tasks.rados.rados.0.smithi138.stdout:576: oid 45 version 233 is already newer than 233 2023-06-27T12:40:30.429 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 233 (ObjNum 279 snap 75 seq_num 279) dirty exists 2023-06-27T12:40:30.429 INFO:tasks.rados.rados.0.smithi138.stdout:576: left oid 45 (ObjNum 279 snap 75 seq_num 279) 2023-06-27T12:40:30.494 INFO:tasks.rados.rados.0.smithi138.stdout:581: expect (ObjNum 226 snap 55 seq_num 226) 2023-06-27T12:40:30.795 INFO:tasks.rados.rados.0.smithi138.stdout:580: finishing write tid 1 to smithi138138488-40 2023-06-27T12:40:30.796 INFO:tasks.rados.rados.0.smithi138.stdout:582: finishing write tid 1 to smithi138138488-19 2023-06-27T12:40:30.796 INFO:tasks.rados.rados.0.smithi138.stdout:582: finishing write tid 2 to smithi138138488-19 2023-06-27T12:40:30.796 INFO:tasks.rados.rados.0.smithi138.stdout:582: finishing write tid 3 to smithi138138488-19 2023-06-27T12:40:30.796 INFO:tasks.rados.rados.0.smithi138.stdout:582: finishing write tid 4 to smithi138138488-19 2023-06-27T12:40:30.796 INFO:tasks.rados.rados.0.smithi138.stdout:582: oid 19 updating version 0 to 171 2023-06-27T12:40:30.796 INFO:tasks.rados.rados.0.smithi138.stdout:582: oid 19 version 171 is already newer than 171 2023-06-27T12:40:30.797 INFO:tasks.rados.rados.0.smithi138.stdout:582: oid 19 version 171 is already newer than 171 2023-06-27T12:40:30.797 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 171 (ObjNum 281 snap 75 seq_num 281) dirty exists 2023-06-27T12:40:30.797 INFO:tasks.rados.rados.0.smithi138.stdout:582: left oid 19 (ObjNum 281 snap 75 seq_num 281) 2023-06-27T12:40:30.797 INFO:tasks.rados.rados.0.smithi138.stdout:580: finishing write tid 2 to smithi138138488-40 2023-06-27T12:40:30.797 INFO:tasks.rados.rados.0.smithi138.stdout:580: finishing write tid 3 to smithi138138488-40 2023-06-27T12:40:30.797 INFO:tasks.rados.rados.0.smithi138.stdout:580: finishing write tid 4 to smithi138138488-40 2023-06-27T12:40:30.797 INFO:tasks.rados.rados.0.smithi138.stdout:580: oid 40 updating version 0 to 64 2023-06-27T12:40:30.797 INFO:tasks.rados.rados.0.smithi138.stdout:580: oid 40 updating version 64 to 67 2023-06-27T12:40:30.798 INFO:tasks.rados.rados.0.smithi138.stdout:580: oid 40 version 67 is already newer than 67 2023-06-27T12:40:30.798 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 67 (ObjNum 280 snap 75 seq_num 280) dirty exists 2023-06-27T12:40:30.798 INFO:tasks.rados.rados.0.smithi138.stdout:580: left oid 40 (ObjNum 280 snap 75 seq_num 280) 2023-06-27T12:40:31.138 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:40:31.264 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:31.259+0000 7f19d4a09700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:31.267 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:31.263+0000 7f19d4a09700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:31.405 INFO:tasks.rados.rados.0.smithi138.stdout:576: done (5 left) 2023-06-27T12:40:31.405 INFO:tasks.rados.rados.0.smithi138.stdout:577: done (4 left) 2023-06-27T12:40:31.406 INFO:tasks.rados.rados.0.smithi138.stdout:580: done (3 left) 2023-06-27T12:40:31.406 INFO:tasks.rados.rados.0.smithi138.stdout:581: done (2 left) 2023-06-27T12:40:31.406 INFO:tasks.rados.rados.0.smithi138.stdout:582: done (1 left) 2023-06-27T12:40:31.406 INFO:tasks.rados.rados.0.smithi138.stdout:583: done (0 left) 2023-06-27T12:40:31.406 INFO:tasks.rados.rados.0.smithi138.stdout:584: write (excl) oid 28 current snap is 76 2023-06-27T12:40:31.406 INFO:tasks.rados.rados.0.smithi138.stdout:584: seq_num 282 ranges {739574=725279,1968628=721093} 2023-06-27T12:40:31.413 INFO:tasks.rados.rados.0.smithi138.stdout:584: writing smithi138138488-28 from 739574 to 1464853 tid 1 2023-06-27T12:40:31.420 INFO:tasks.rados.rados.0.smithi138.stdout:584: writing smithi138138488-28 from 1968628 to 2689721 tid 2 2023-06-27T12:40:31.420 INFO:tasks.rados.rados.0.smithi138.stdout:585: snap_create 2023-06-27T12:40:31.484 INFO:tasks.rados.rados.0.smithi138.stdout:584: finishing write tid 1 to smithi138138488-28 2023-06-27T12:40:31.538 INFO:tasks.rados.rados.0.smithi138.stdout:584: finishing write tid 2 to smithi138138488-28 2023-06-27T12:40:31.538 INFO:tasks.rados.rados.0.smithi138.stdout:584: finishing write tid 3 to smithi138138488-28 2023-06-27T12:40:31.539 INFO:tasks.rados.rados.0.smithi138.stdout:584: finishing write tid 4 to smithi138138488-28 2023-06-27T12:40:31.539 INFO:tasks.rados.rados.0.smithi138.stdout:584: oid 28 updating version 0 to 105 2023-06-27T12:40:31.539 INFO:tasks.rados.rados.0.smithi138.stdout:584: oid 28 version 105 is already newer than 105 2023-06-27T12:40:31.539 INFO:tasks.rados.rados.0.smithi138.stdout:584: oid 28 version 105 is already newer than 102 2023-06-27T12:40:31.539 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 105 (ObjNum 282 snap 76 seq_num 282) dirty exists 2023-06-27T12:40:31.540 INFO:tasks.rados.rados.0.smithi138.stdout:584: left oid 28 (ObjNum 282 snap 76 seq_num 282) 2023-06-27T12:40:32.410 INFO:tasks.rados.rados.0.smithi138.stdout:584: done (1 left) 2023-06-27T12:40:32.410 INFO:tasks.rados.rados.0.smithi138.stdout:585: done (0 left) 2023-06-27T12:40:32.410 INFO:tasks.rados.rados.0.smithi138.stdout:586: write (excl) oid 19 current snap is 77 2023-06-27T12:40:32.411 INFO:tasks.rados.rados.0.smithi138.stdout:586: seq_num 283 ranges {756381=400528,1910757=464751} 2023-06-27T12:40:32.416 INFO:tasks.rados.rados.0.smithi138.stdout:586: writing smithi138138488-19 from 756381 to 1156909 tid 1 2023-06-27T12:40:32.423 INFO:tasks.rados.rados.0.smithi138.stdout:586: writing smithi138138488-19 from 1910757 to 2375508 tid 2 2023-06-27T12:40:32.423 INFO:tasks.rados.rados.0.smithi138.stdout:587: snap_remove snap 72 2023-06-27T12:40:32.831 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:32.832 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-06-27T12:40:32.973 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:32.964+0000 7f0c56248700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:32.976 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:32.973+0000 7f0c56248700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:33.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:40:33.416 INFO:tasks.rados.rados.0.smithi138.stdout:587: done (1 left) 2023-06-27T12:40:33.417 INFO:tasks.rados.rados.0.smithi138.stdout:588: write (excl) oid 50 current snap is 77 2023-06-27T12:40:33.417 INFO:tasks.rados.rados.0.smithi138.stdout:586: finishing write tid 1 to smithi138138488-19 2023-06-27T12:40:33.417 INFO:tasks.rados.rados.0.smithi138.stdout:588: seq_num 284 ranges {773188=475777,1969239=724762} 2023-06-27T12:40:33.419 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 1 (10000) 2023-06-27T12:40:33.425 INFO:tasks.rados.rados.0.smithi138.stdout:588: writing smithi138138488-50 from 773188 to 1248965 tid 1 2023-06-27T12:40:33.437 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-06-27T12:40:33.438 INFO:tasks.rados.rados.0.smithi138.stdout:588: writing smithi138138488-50 from 1969239 to 2694001 tid 2 2023-06-27T12:40:33.439 INFO:tasks.rados.rados.0.smithi138.stdout:589: snap_create 2023-06-27T12:40:33.439 INFO:tasks.rados.rados.0.smithi138.stdout:586: finishing write tid 2 to smithi138138488-19 2023-06-27T12:40:33.439 INFO:tasks.rados.rados.0.smithi138.stdout:586: finishing write tid 3 to smithi138138488-19 2023-06-27T12:40:33.440 INFO:tasks.rados.rados.0.smithi138.stdout:586: finishing write tid 4 to smithi138138488-19 2023-06-27T12:40:33.440 INFO:tasks.rados.rados.0.smithi138.stdout:586: oid 19 updating version 0 to 175 2023-06-27T12:40:33.440 INFO:tasks.rados.rados.0.smithi138.stdout:586: oid 19 version 175 is already newer than 175 2023-06-27T12:40:33.440 INFO:tasks.rados.rados.0.smithi138.stdout:586: oid 19 version 175 is already newer than 172 2023-06-27T12:40:33.440 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 175 (ObjNum 283 snap 77 seq_num 283) dirty exists 2023-06-27T12:40:33.440 INFO:tasks.rados.rados.0.smithi138.stdout:586: left oid 19 (ObjNum 283 snap 77 seq_num 283) 2023-06-27T12:40:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:33.479+0000 7f4a2805b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:33.484+0000 7f4a2805b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:33.511 INFO:tasks.rados.rados.0.smithi138.stdout:588: finishing write tid 1 to smithi138138488-50 2023-06-27T12:40:33.570 INFO:tasks.rados.rados.0.smithi138.stdout:588: finishing write tid 2 to smithi138138488-50 2023-06-27T12:40:33.570 INFO:tasks.rados.rados.0.smithi138.stdout:588: finishing write tid 3 to smithi138138488-50 2023-06-27T12:40:33.571 INFO:tasks.rados.rados.0.smithi138.stdout:588: finishing write tid 4 to smithi138138488-50 2023-06-27T12:40:33.571 INFO:tasks.rados.rados.0.smithi138.stdout:588: oid 50 updating version 0 to 43 2023-06-27T12:40:33.571 INFO:tasks.rados.rados.0.smithi138.stdout:588: oid 50 updating version 43 to 46 2023-06-27T12:40:33.572 INFO:tasks.rados.rados.0.smithi138.stdout:588: oid 50 version 46 is already newer than 46 2023-06-27T12:40:33.572 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 46 (ObjNum 284 snap 77 seq_num 284) dirty exists 2023-06-27T12:40:33.572 INFO:tasks.rados.rados.0.smithi138.stdout:588: left oid 50 (ObjNum 284 snap 77 seq_num 284) 2023-06-27T12:40:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:33.578+0000 7f2ed0c0d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:33.587+0000 7f2ed0c0d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:34.420 INFO:tasks.rados.rados.0.smithi138.stdout:586: done (2 left) 2023-06-27T12:40:34.420 INFO:tasks.rados.rados.0.smithi138.stdout:588: done (1 left) 2023-06-27T12:40:34.420 INFO:tasks.rados.rados.0.smithi138.stdout:589: done (0 left) 2023-06-27T12:40:34.420 INFO:tasks.rados.rados.0.smithi138.stdout:590: write (excl) oid 2 current snap is 78 2023-06-27T12:40:34.421 INFO:tasks.rados.rados.0.smithi138.stdout:590: seq_num 285 ranges {789995=551026,2027721=468420} 2023-06-27T12:40:34.423 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.1 to 1 (10000) 2023-06-27T12:40:34.427 INFO:tasks.rados.rados.0.smithi138.stdout:590: writing smithi138138488-2 from 789995 to 1341021 tid 1 2023-06-27T12:40:34.434 INFO:tasks.rados.rados.0.smithi138.stdout:590: writing smithi138138488-2 from 2027721 to 2496141 tid 2 2023-06-27T12:40:34.434 INFO:tasks.rados.rados.0.smithi138.stdout:591: read oid 35 snap -1 2023-06-27T12:40:34.434 INFO:tasks.rados.rados.0.smithi138.stdout:591: expect (ObjNum 271 snap 70 seq_num 271) 2023-06-27T12:40:34.435 INFO:tasks.rados.rados.0.smithi138.stdout:592: delete oid 8 current snap is 78 2023-06-27T12:40:34.435 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-06-27T12:40:34.567 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:34.558+0000 7f8de4780700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:34.566+0000 7f8de4780700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:35.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-06-27T12:40:35.424 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.2 to 1 (10000) 2023-06-27T12:40:35.437 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-06-27T12:40:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:35.558+0000 7f69d0e5e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:35.567+0000 7f69d0e5e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:35.593 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:40:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:35.715+0000 7f2e27df0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:35.719+0000 7f2e27df0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:36.430 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.3 to 1 (10000) 2023-06-27T12:40:36.443 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-06-27T12:40:36.444 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:36.577+0000 7faf37792700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:36.585+0000 7faf37792700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:36.787 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:36.789 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:36.801 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2023-06-27T12:40:36.423010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105181055,"num_objects":198,"num_object_clones":140,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":979,"num_read_kb":624465,"num_write":1102,"num_write_kb":280937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":691752,"kb_used_data":568072,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376795608,"statfs":{"total":386547056640,"available":385838702592,"internally_reserved":0,"allocated":581705728,"data_stored":676819668,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1428496,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":8877,"num_write":20,"num_write_kb":5555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019348"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":319,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.817092+0000","last_change":"2023-06-27T12:40:33.819159+0000","last_active":"2023-06-27T12:40:33.817092+0000","last_peered":"2023-06-27T12:40:33.817092+0000","last_clean":"2023-06-27T12:40:33.817092+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:33.817092+0000","last_undegraded":"2023-06-27T12:40:33.817092+0000","last_fullsized":"2023-06-27T12:40:33.817092+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":435,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.430347+0000","last_change":"2023-06-27T12:40:33.439175+0000","last_active":"2023-06-27T12:40:33.430347+0000","last_peered":"2023-06-27T12:40:33.430347+0000","last_clean":"2023-06-27T12:40:33.430347+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:33.430347+0000","last_undegraded":"2023-06-27T12:40:33.430347+0000","last_fullsized":"2023-06-27T12:40:33.430347+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":438,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.455326+0000","last_change":"2023-06-27T12:40:33.461948+0000","last_active":"2023-06-27T12:40:33.455326+0000","last_peered":"2023-06-27T12:40:33.455326+0000","last_clean":"2023-06-27T12:40:33.455326+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:33.455326+0000","last_undegraded":"2023-06-27T12:40:33.455326+0000","last_fullsized":"2023-06-27T12:40:33.455326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'67","reported_seq":402,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.823827+0000","last_change":"2023-06-27T12:40:33.824008+0000","last_active":"2023-06-27T12:40:33.823827+0000","last_peered":"2023-06-27T12:40:33.823827+0000","last_clean":"2023-06-27T12:40:33.823827+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:33.823827+0000","last_undegraded":"2023-06-27T12:40:33.823827+0000","last_fullsized":"2023-06-27T12:40:33.823827+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050553,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":27199,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":348,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.454098+0000","last_change":"2023-06-27T12:40:27.808791+0000","last_active":"2023-06-27T12:40:28.454098+0000","last_peered":"2023-06-27T12:40:28.454098+0000","last_clean":"2023-06-27T12:40:28.454098+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:28.454098+0000","last_undegraded":"2023-06-27T12:40:28.454098+0000","last_fullsized":"2023-06-27T12:40:28.454098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":493,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.814624+0000","last_change":"2023-06-27T12:40:33.818274+0000","last_active":"2023-06-27T12:40:33.814624+0000","last_peered":"2023-06-27T12:40:33.814624+0000","last_clean":"2023-06-27T12:40:33.814624+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:33.814624+0000","last_undegraded":"2023-06-27T12:40:33.814624+0000","last_fullsized":"2023-06-27T12:40:33.814624+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":113812,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"163'105","reported_seq":402,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.811455+0000","last_change":"2023-06-27T12:40:33.883382+0000","last_active":"2023-06-27T12:40:33.811455+0000","last_peered":"2023-06-27T12:40:33.811455+0000","last_clean":"2023-06-27T12:40:33.811455+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:33.811455+0000","last_undegraded":"2023-06-27T12:40:33.811455+0000","last_fullsized":"2023-06-27T12:40:33.811455+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227043,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":57,"num_read_kb":30118,"num_write":63,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":364,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.511273+0000","last_change":"2023-06-27T12:40:33.451340+0000","last_active":"2023-06-27T12:40:33.511273+0000","last_peered":"2023-06-27T12:40:33.511273+0000","last_clean":"2023-06-27T12:40:33.511273+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:33.511273+0000","last_undegraded":"2023-06-27T12:40:33.511273+0000","last_fullsized":"2023-06-27T12:40:33.511273+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2773188,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.432109+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:33.432109+0000","last_peered":"2023-06-27T12:40:33.432109+0000","last_clean":"2023-06-27T12:40:33.432109+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:33.432109+0000","last_undegraded":"2023-06-27T12:40:33.432109+0000","last_fullsized":"2023-06-27T12:40:33.432109+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":312,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.452728+0000","last_change":"2023-06-27T12:40:33.461555+0000","last_active":"2023-06-27T12:40:33.452728+0000","last_peered":"2023-06-27T12:40:33.452728+0000","last_clean":"2023-06-27T12:40:33.452728+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:33.452728+0000","last_undegraded":"2023-06-27T12:40:33.452728+0000","last_fullsized":"2023-06-27T12:40:33.452728+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":249,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.814535+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:33.814535+0000","last_peered":"2023-06-27T12:40:33.814535+0000","last_clean":"2023-06-27T12:40:33.814535+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:33.814535+0000","last_undegraded":"2023-06-27T12:40:33.814535+0000","last_fullsized":"2023-06-27T12:40:33.814535+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"158'76","reported_seq":391,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.427040+0000","last_change":"2023-06-27T12:40:33.438763+0000","last_active":"2023-06-27T12:40:33.427040+0000","last_peered":"2023-06-27T12:40:33.427040+0000","last_clean":"2023-06-27T12:40:33.427040+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:33.427040+0000","last_undegraded":"2023-06-27T12:40:33.427040+0000","last_fullsized":"2023-06-27T12:40:33.427040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689285,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":36,"num_read_kb":31324,"num_write":50,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.448377+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:33.448377+0000","last_peered":"2023-06-27T12:40:33.448377+0000","last_clean":"2023-06-27T12:40:33.448377+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:33.448377+0000","last_undegraded":"2023-06-27T12:40:33.448377+0000","last_fullsized":"2023-06-27T12:40:33.448377+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"160'233","reported_seq":486,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.533100+0000","last_change":"2023-06-27T12:40:28.538999+0000","last_active":"2023-06-27T12:40:28.533100+0000","last_peered":"2023-06-27T12:40:28.533100+0000","last_clean":"2023-06-27T12:40:28.533100+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:28.533100+0000","last_undegraded":"2023-06-27T12:40:28.533100+0000","last_fullsized":"2023-06-27T12:40:28.533100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10319597,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":71887,"num_write":132,"num_write_kb":39413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":247,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.445531+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:33.445531+0000","last_peered":"2023-06-27T12:40:33.445531+0000","last_clean":"2023-06-27T12:40:33.445531+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:33.445531+0000","last_undegraded":"2023-06-27T12:40:33.445531+0000","last_fullsized":"2023-06-27T12:40:33.445531+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":372,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.449489+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:28.449489+0000","last_peered":"2023-06-27T12:40:28.449489+0000","last_clean":"2023-06-27T12:40:28.449489+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:28.449489+0000","last_undegraded":"2023-06-27T12:40:28.449489+0000","last_fullsized":"2023-06-27T12:40:28.449489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"164'175","reported_seq":498,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.450797+0000","last_change":"2023-06-27T12:40:33.450942+0000","last_active":"2023-06-27T12:40:33.450797+0000","last_peered":"2023-06-27T12:40:33.450797+0000","last_clean":"2023-06-27T12:40:33.450797+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:33.450797+0000","last_undegraded":"2023-06-27T12:40:33.450797+0000","last_fullsized":"2023-06-27T12:40:33.450797+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15025524,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":70272,"num_write":126,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":249,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.815337+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:33.815337+0000","last_peered":"2023-06-27T12:40:33.815337+0000","last_clean":"2023-06-27T12:40:33.815337+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:33.815337+0000","last_undegraded":"2023-06-27T12:40:33.815337+0000","last_fullsized":"2023-06-27T12:40:33.815337+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"151'73","reported_seq":376,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.446866+0000","last_change":"2023-06-27T12:40:27.809749+0000","last_active":"2023-06-27T12:40:28.446866+0000","last_peered":"2023-06-27T12:40:28.446866+0000","last_clean":"2023-06-27T12:40:28.446866+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:28.446866+0000","last_undegraded":"2023-06-27T12:40:28.446866+0000","last_fullsized":"2023-06-27T12:40:28.446866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840548,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":33,"num_read_kb":24675,"num_write":53,"num_write_kb":15374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.447682+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:28.447682+0000","last_peered":"2023-06-27T12:40:28.447682+0000","last_clean":"2023-06-27T12:40:28.447682+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:28.447682+0000","last_undegraded":"2023-06-27T12:40:28.447682+0000","last_fullsized":"2023-06-27T12:40:28.447682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":413,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.428957+0000","last_change":"2023-06-27T12:40:33.436804+0000","last_active":"2023-06-27T12:40:33.428957+0000","last_peered":"2023-06-27T12:40:33.428957+0000","last_clean":"2023-06-27T12:40:33.428957+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:33.428957+0000","last_undegraded":"2023-06-27T12:40:33.428957+0000","last_fullsized":"2023-06-27T12:40:33.428957+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":355,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.455948+0000","last_change":"2023-06-27T12:40:27.810261+0000","last_active":"2023-06-27T12:40:28.455948+0000","last_peered":"2023-06-27T12:40:28.455948+0000","last_clean":"2023-06-27T12:40:28.455948+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:28.455948+0000","last_undegraded":"2023-06-27T12:40:28.455948+0000","last_fullsized":"2023-06-27T12:40:28.455948+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":255,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.428088+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:33.428088+0000","last_peered":"2023-06-27T12:40:33.428088+0000","last_clean":"2023-06-27T12:40:33.428088+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:33.428088+0000","last_undegraded":"2023-06-27T12:40:33.428088+0000","last_fullsized":"2023-06-27T12:40:33.428088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":247,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.444346+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:33.444346+0000","last_peered":"2023-06-27T12:40:33.444346+0000","last_clean":"2023-06-27T12:40:33.444346+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:33.444346+0000","last_undegraded":"2023-06-27T12:40:33.444346+0000","last_fullsized":"2023-06-27T12:40:33.444346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":411,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.431181+0000","last_change":"2023-06-27T12:40:33.436080+0000","last_active":"2023-06-27T12:40:33.431181+0000","last_peered":"2023-06-27T12:40:33.431181+0000","last_clean":"2023-06-27T12:40:33.431181+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:33.431181+0000","last_undegraded":"2023-06-27T12:40:33.431181+0000","last_fullsized":"2023-06-27T12:40:33.431181+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":42938,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":104590687,"num_objects":196,"num_object_clones":140,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":876,"num_read_kb":624374,"num_write":965,"num_write_kb":279043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673120,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160440,"kb_used_data":129520,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211400,"statfs":{"total":96636764160,"available":96472473600,"internally_reserved":0,"allocated":132628480,"data_stored":154633646,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705825,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167760,"kb_used_data":136840,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204080,"statfs":{"total":96636764160,"available":96464977920,"internally_reserved":0,"allocated":140124160,"data_stored":162201537,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673120,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":182272,"kb_used_data":151352,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94189568,"statfs":{"total":96636764160,"available":96450117632,"internally_reserved":0,"allocated":154984448,"data_stored":182944933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:36.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:36.930 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:36.922+0000 7fa9fb7c3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:36.932 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:36.928+0000 7fa9fb7c3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:37.126 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:37.128 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:37.144 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2023-06-27T12:40:36.423010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105181055,"num_objects":198,"num_object_clones":140,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":979,"num_read_kb":624465,"num_write":1102,"num_write_kb":280937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":691752,"kb_used_data":568072,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376795608,"statfs":{"total":386547056640,"available":385838702592,"internally_reserved":0,"allocated":581705728,"data_stored":676819668,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1428496,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":8877,"num_write":20,"num_write_kb":5555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019348"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":319,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.817092+0000","last_change":"2023-06-27T12:40:33.819159+0000","last_active":"2023-06-27T12:40:33.817092+0000","last_peered":"2023-06-27T12:40:33.817092+0000","last_clean":"2023-06-27T12:40:33.817092+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:33.817092+0000","last_undegraded":"2023-06-27T12:40:33.817092+0000","last_fullsized":"2023-06-27T12:40:33.817092+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":435,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.430347+0000","last_change":"2023-06-27T12:40:33.439175+0000","last_active":"2023-06-27T12:40:33.430347+0000","last_peered":"2023-06-27T12:40:33.430347+0000","last_clean":"2023-06-27T12:40:33.430347+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:33.430347+0000","last_undegraded":"2023-06-27T12:40:33.430347+0000","last_fullsized":"2023-06-27T12:40:33.430347+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":438,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.455326+0000","last_change":"2023-06-27T12:40:33.461948+0000","last_active":"2023-06-27T12:40:33.455326+0000","last_peered":"2023-06-27T12:40:33.455326+0000","last_clean":"2023-06-27T12:40:33.455326+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:33.455326+0000","last_undegraded":"2023-06-27T12:40:33.455326+0000","last_fullsized":"2023-06-27T12:40:33.455326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'67","reported_seq":402,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.823827+0000","last_change":"2023-06-27T12:40:33.824008+0000","last_active":"2023-06-27T12:40:33.823827+0000","last_peered":"2023-06-27T12:40:33.823827+0000","last_clean":"2023-06-27T12:40:33.823827+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:33.823827+0000","last_undegraded":"2023-06-27T12:40:33.823827+0000","last_fullsized":"2023-06-27T12:40:33.823827+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050553,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":27199,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":348,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.454098+0000","last_change":"2023-06-27T12:40:27.808791+0000","last_active":"2023-06-27T12:40:28.454098+0000","last_peered":"2023-06-27T12:40:28.454098+0000","last_clean":"2023-06-27T12:40:28.454098+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:28.454098+0000","last_undegraded":"2023-06-27T12:40:28.454098+0000","last_fullsized":"2023-06-27T12:40:28.454098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":493,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.814624+0000","last_change":"2023-06-27T12:40:33.818274+0000","last_active":"2023-06-27T12:40:33.814624+0000","last_peered":"2023-06-27T12:40:33.814624+0000","last_clean":"2023-06-27T12:40:33.814624+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:33.814624+0000","last_undegraded":"2023-06-27T12:40:33.814624+0000","last_fullsized":"2023-06-27T12:40:33.814624+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":113812,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"163'105","reported_seq":402,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.811455+0000","last_change":"2023-06-27T12:40:33.883382+0000","last_active":"2023-06-27T12:40:33.811455+0000","last_peered":"2023-06-27T12:40:33.811455+0000","last_clean":"2023-06-27T12:40:33.811455+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:33.811455+0000","last_undegraded":"2023-06-27T12:40:33.811455+0000","last_fullsized":"2023-06-27T12:40:33.811455+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227043,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":57,"num_read_kb":30118,"num_write":63,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":364,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.511273+0000","last_change":"2023-06-27T12:40:33.451340+0000","last_active":"2023-06-27T12:40:33.511273+0000","last_peered":"2023-06-27T12:40:33.511273+0000","last_clean":"2023-06-27T12:40:33.511273+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:33.511273+0000","last_undegraded":"2023-06-27T12:40:33.511273+0000","last_fullsized":"2023-06-27T12:40:33.511273+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2773188,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.432109+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:33.432109+0000","last_peered":"2023-06-27T12:40:33.432109+0000","last_clean":"2023-06-27T12:40:33.432109+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:33.432109+0000","last_undegraded":"2023-06-27T12:40:33.432109+0000","last_fullsized":"2023-06-27T12:40:33.432109+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":312,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.452728+0000","last_change":"2023-06-27T12:40:33.461555+0000","last_active":"2023-06-27T12:40:33.452728+0000","last_peered":"2023-06-27T12:40:33.452728+0000","last_clean":"2023-06-27T12:40:33.452728+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:33.452728+0000","last_undegraded":"2023-06-27T12:40:33.452728+0000","last_fullsized":"2023-06-27T12:40:33.452728+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":249,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.814535+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:33.814535+0000","last_peered":"2023-06-27T12:40:33.814535+0000","last_clean":"2023-06-27T12:40:33.814535+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:33.814535+0000","last_undegraded":"2023-06-27T12:40:33.814535+0000","last_fullsized":"2023-06-27T12:40:33.814535+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"158'76","reported_seq":391,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.427040+0000","last_change":"2023-06-27T12:40:33.438763+0000","last_active":"2023-06-27T12:40:33.427040+0000","last_peered":"2023-06-27T12:40:33.427040+0000","last_clean":"2023-06-27T12:40:33.427040+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:33.427040+0000","last_undegraded":"2023-06-27T12:40:33.427040+0000","last_fullsized":"2023-06-27T12:40:33.427040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689285,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":36,"num_read_kb":31324,"num_write":50,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.448377+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:33.448377+0000","last_peered":"2023-06-27T12:40:33.448377+0000","last_clean":"2023-06-27T12:40:33.448377+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:33.448377+0000","last_undegraded":"2023-06-27T12:40:33.448377+0000","last_fullsized":"2023-06-27T12:40:33.448377+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"160'233","reported_seq":486,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.533100+0000","last_change":"2023-06-27T12:40:28.538999+0000","last_active":"2023-06-27T12:40:28.533100+0000","last_peered":"2023-06-27T12:40:28.533100+0000","last_clean":"2023-06-27T12:40:28.533100+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:28.533100+0000","last_undegraded":"2023-06-27T12:40:28.533100+0000","last_fullsized":"2023-06-27T12:40:28.533100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10319597,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":71887,"num_write":132,"num_write_kb":39413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":247,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.445531+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:33.445531+0000","last_peered":"2023-06-27T12:40:33.445531+0000","last_clean":"2023-06-27T12:40:33.445531+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:33.445531+0000","last_undegraded":"2023-06-27T12:40:33.445531+0000","last_fullsized":"2023-06-27T12:40:33.445531+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":372,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.449489+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:28.449489+0000","last_peered":"2023-06-27T12:40:28.449489+0000","last_clean":"2023-06-27T12:40:28.449489+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:28.449489+0000","last_undegraded":"2023-06-27T12:40:28.449489+0000","last_fullsized":"2023-06-27T12:40:28.449489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"164'175","reported_seq":498,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.450797+0000","last_change":"2023-06-27T12:40:33.450942+0000","last_active":"2023-06-27T12:40:33.450797+0000","last_peered":"2023-06-27T12:40:33.450797+0000","last_clean":"2023-06-27T12:40:33.450797+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:33.450797+0000","last_undegraded":"2023-06-27T12:40:33.450797+0000","last_fullsized":"2023-06-27T12:40:33.450797+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15025524,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":70272,"num_write":126,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":249,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.815337+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:33.815337+0000","last_peered":"2023-06-27T12:40:33.815337+0000","last_clean":"2023-06-27T12:40:33.815337+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:33.815337+0000","last_undegraded":"2023-06-27T12:40:33.815337+0000","last_fullsized":"2023-06-27T12:40:33.815337+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"151'73","reported_seq":376,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.446866+0000","last_change":"2023-06-27T12:40:27.809749+0000","last_active":"2023-06-27T12:40:28.446866+0000","last_peered":"2023-06-27T12:40:28.446866+0000","last_clean":"2023-06-27T12:40:28.446866+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:28.446866+0000","last_undegraded":"2023-06-27T12:40:28.446866+0000","last_fullsized":"2023-06-27T12:40:28.446866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840548,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":33,"num_read_kb":24675,"num_write":53,"num_write_kb":15374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.447682+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:28.447682+0000","last_peered":"2023-06-27T12:40:28.447682+0000","last_clean":"2023-06-27T12:40:28.447682+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:28.447682+0000","last_undegraded":"2023-06-27T12:40:28.447682+0000","last_fullsized":"2023-06-27T12:40:28.447682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":413,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.428957+0000","last_change":"2023-06-27T12:40:33.436804+0000","last_active":"2023-06-27T12:40:33.428957+0000","last_peered":"2023-06-27T12:40:33.428957+0000","last_clean":"2023-06-27T12:40:33.428957+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:33.428957+0000","last_undegraded":"2023-06-27T12:40:33.428957+0000","last_fullsized":"2023-06-27T12:40:33.428957+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":355,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.455948+0000","last_change":"2023-06-27T12:40:27.810261+0000","last_active":"2023-06-27T12:40:28.455948+0000","last_peered":"2023-06-27T12:40:28.455948+0000","last_clean":"2023-06-27T12:40:28.455948+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:28.455948+0000","last_undegraded":"2023-06-27T12:40:28.455948+0000","last_fullsized":"2023-06-27T12:40:28.455948+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":255,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.428088+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:33.428088+0000","last_peered":"2023-06-27T12:40:33.428088+0000","last_clean":"2023-06-27T12:40:33.428088+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:33.428088+0000","last_undegraded":"2023-06-27T12:40:33.428088+0000","last_fullsized":"2023-06-27T12:40:33.428088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":247,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.444346+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:33.444346+0000","last_peered":"2023-06-27T12:40:33.444346+0000","last_clean":"2023-06-27T12:40:33.444346+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:33.444346+0000","last_undegraded":"2023-06-27T12:40:33.444346+0000","last_fullsized":"2023-06-27T12:40:33.444346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":411,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.431181+0000","last_change":"2023-06-27T12:40:33.436080+0000","last_active":"2023-06-27T12:40:33.431181+0000","last_peered":"2023-06-27T12:40:33.431181+0000","last_clean":"2023-06-27T12:40:33.431181+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:33.431181+0000","last_undegraded":"2023-06-27T12:40:33.431181+0000","last_fullsized":"2023-06-27T12:40:33.431181+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":42938,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":104590687,"num_objects":196,"num_object_clones":140,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":876,"num_read_kb":624374,"num_write":965,"num_write_kb":279043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673120,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160440,"kb_used_data":129520,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211400,"statfs":{"total":96636764160,"available":96472473600,"internally_reserved":0,"allocated":132628480,"data_stored":154633646,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705825,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167760,"kb_used_data":136840,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204080,"statfs":{"total":96636764160,"available":96464977920,"internally_reserved":0,"allocated":140124160,"data_stored":162201537,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673120,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":182272,"kb_used_data":151352,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94189568,"statfs":{"total":96636764160,"available":96450117632,"internally_reserved":0,"allocated":154984448,"data_stored":182944933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:37.147 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-06-27T12:40:37.148 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:37.149 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:37.277 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:37.269+0000 7ff6ba561700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:37.280 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:37.276+0000 7ff6ba561700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:37.472 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:37.474 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:37.486 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2023-06-27T12:40:36.423010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105181055,"num_objects":198,"num_object_clones":140,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":979,"num_read_kb":624465,"num_write":1102,"num_write_kb":280937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":691752,"kb_used_data":568072,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376795608,"statfs":{"total":386547056640,"available":385838702592,"internally_reserved":0,"allocated":581705728,"data_stored":676819668,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1428496,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":8877,"num_write":20,"num_write_kb":5555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019348"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":319,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.817092+0000","last_change":"2023-06-27T12:40:33.819159+0000","last_active":"2023-06-27T12:40:33.817092+0000","last_peered":"2023-06-27T12:40:33.817092+0000","last_clean":"2023-06-27T12:40:33.817092+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:33.817092+0000","last_undegraded":"2023-06-27T12:40:33.817092+0000","last_fullsized":"2023-06-27T12:40:33.817092+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":435,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.430347+0000","last_change":"2023-06-27T12:40:33.439175+0000","last_active":"2023-06-27T12:40:33.430347+0000","last_peered":"2023-06-27T12:40:33.430347+0000","last_clean":"2023-06-27T12:40:33.430347+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:33.430347+0000","last_undegraded":"2023-06-27T12:40:33.430347+0000","last_fullsized":"2023-06-27T12:40:33.430347+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":438,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.455326+0000","last_change":"2023-06-27T12:40:33.461948+0000","last_active":"2023-06-27T12:40:33.455326+0000","last_peered":"2023-06-27T12:40:33.455326+0000","last_clean":"2023-06-27T12:40:33.455326+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:33.455326+0000","last_undegraded":"2023-06-27T12:40:33.455326+0000","last_fullsized":"2023-06-27T12:40:33.455326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'67","reported_seq":402,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.823827+0000","last_change":"2023-06-27T12:40:33.824008+0000","last_active":"2023-06-27T12:40:33.823827+0000","last_peered":"2023-06-27T12:40:33.823827+0000","last_clean":"2023-06-27T12:40:33.823827+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:33.823827+0000","last_undegraded":"2023-06-27T12:40:33.823827+0000","last_fullsized":"2023-06-27T12:40:33.823827+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050553,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":27199,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":348,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.454098+0000","last_change":"2023-06-27T12:40:27.808791+0000","last_active":"2023-06-27T12:40:28.454098+0000","last_peered":"2023-06-27T12:40:28.454098+0000","last_clean":"2023-06-27T12:40:28.454098+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:28.454098+0000","last_undegraded":"2023-06-27T12:40:28.454098+0000","last_fullsized":"2023-06-27T12:40:28.454098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":493,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.814624+0000","last_change":"2023-06-27T12:40:33.818274+0000","last_active":"2023-06-27T12:40:33.814624+0000","last_peered":"2023-06-27T12:40:33.814624+0000","last_clean":"2023-06-27T12:40:33.814624+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:33.814624+0000","last_undegraded":"2023-06-27T12:40:33.814624+0000","last_fullsized":"2023-06-27T12:40:33.814624+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":113812,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"163'105","reported_seq":402,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.811455+0000","last_change":"2023-06-27T12:40:33.883382+0000","last_active":"2023-06-27T12:40:33.811455+0000","last_peered":"2023-06-27T12:40:33.811455+0000","last_clean":"2023-06-27T12:40:33.811455+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:33.811455+0000","last_undegraded":"2023-06-27T12:40:33.811455+0000","last_fullsized":"2023-06-27T12:40:33.811455+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227043,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":57,"num_read_kb":30118,"num_write":63,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":364,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.511273+0000","last_change":"2023-06-27T12:40:33.451340+0000","last_active":"2023-06-27T12:40:33.511273+0000","last_peered":"2023-06-27T12:40:33.511273+0000","last_clean":"2023-06-27T12:40:33.511273+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:33.511273+0000","last_undegraded":"2023-06-27T12:40:33.511273+0000","last_fullsized":"2023-06-27T12:40:33.511273+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2773188,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.432109+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:33.432109+0000","last_peered":"2023-06-27T12:40:33.432109+0000","last_clean":"2023-06-27T12:40:33.432109+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:33.432109+0000","last_undegraded":"2023-06-27T12:40:33.432109+0000","last_fullsized":"2023-06-27T12:40:33.432109+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":312,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.452728+0000","last_change":"2023-06-27T12:40:33.461555+0000","last_active":"2023-06-27T12:40:33.452728+0000","last_peered":"2023-06-27T12:40:33.452728+0000","last_clean":"2023-06-27T12:40:33.452728+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:33.452728+0000","last_undegraded":"2023-06-27T12:40:33.452728+0000","last_fullsized":"2023-06-27T12:40:33.452728+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":249,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.814535+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:33.814535+0000","last_peered":"2023-06-27T12:40:33.814535+0000","last_clean":"2023-06-27T12:40:33.814535+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:33.814535+0000","last_undegraded":"2023-06-27T12:40:33.814535+0000","last_fullsized":"2023-06-27T12:40:33.814535+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"158'76","reported_seq":391,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.427040+0000","last_change":"2023-06-27T12:40:33.438763+0000","last_active":"2023-06-27T12:40:33.427040+0000","last_peered":"2023-06-27T12:40:33.427040+0000","last_clean":"2023-06-27T12:40:33.427040+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:33.427040+0000","last_undegraded":"2023-06-27T12:40:33.427040+0000","last_fullsized":"2023-06-27T12:40:33.427040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689285,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":36,"num_read_kb":31324,"num_write":50,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.448377+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:33.448377+0000","last_peered":"2023-06-27T12:40:33.448377+0000","last_clean":"2023-06-27T12:40:33.448377+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:33.448377+0000","last_undegraded":"2023-06-27T12:40:33.448377+0000","last_fullsized":"2023-06-27T12:40:33.448377+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"160'233","reported_seq":486,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.533100+0000","last_change":"2023-06-27T12:40:28.538999+0000","last_active":"2023-06-27T12:40:28.533100+0000","last_peered":"2023-06-27T12:40:28.533100+0000","last_clean":"2023-06-27T12:40:28.533100+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:28.533100+0000","last_undegraded":"2023-06-27T12:40:28.533100+0000","last_fullsized":"2023-06-27T12:40:28.533100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10319597,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":71887,"num_write":132,"num_write_kb":39413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":247,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.445531+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:33.445531+0000","last_peered":"2023-06-27T12:40:33.445531+0000","last_clean":"2023-06-27T12:40:33.445531+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:33.445531+0000","last_undegraded":"2023-06-27T12:40:33.445531+0000","last_fullsized":"2023-06-27T12:40:33.445531+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":372,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.449489+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:28.449489+0000","last_peered":"2023-06-27T12:40:28.449489+0000","last_clean":"2023-06-27T12:40:28.449489+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:28.449489+0000","last_undegraded":"2023-06-27T12:40:28.449489+0000","last_fullsized":"2023-06-27T12:40:28.449489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"164'175","reported_seq":498,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.450797+0000","last_change":"2023-06-27T12:40:33.450942+0000","last_active":"2023-06-27T12:40:33.450797+0000","last_peered":"2023-06-27T12:40:33.450797+0000","last_clean":"2023-06-27T12:40:33.450797+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:33.450797+0000","last_undegraded":"2023-06-27T12:40:33.450797+0000","last_fullsized":"2023-06-27T12:40:33.450797+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15025524,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":70272,"num_write":126,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":249,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.815337+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:33.815337+0000","last_peered":"2023-06-27T12:40:33.815337+0000","last_clean":"2023-06-27T12:40:33.815337+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:33.815337+0000","last_undegraded":"2023-06-27T12:40:33.815337+0000","last_fullsized":"2023-06-27T12:40:33.815337+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"151'73","reported_seq":376,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.446866+0000","last_change":"2023-06-27T12:40:27.809749+0000","last_active":"2023-06-27T12:40:28.446866+0000","last_peered":"2023-06-27T12:40:28.446866+0000","last_clean":"2023-06-27T12:40:28.446866+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:28.446866+0000","last_undegraded":"2023-06-27T12:40:28.446866+0000","last_fullsized":"2023-06-27T12:40:28.446866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840548,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":33,"num_read_kb":24675,"num_write":53,"num_write_kb":15374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.447682+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:28.447682+0000","last_peered":"2023-06-27T12:40:28.447682+0000","last_clean":"2023-06-27T12:40:28.447682+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:28.447682+0000","last_undegraded":"2023-06-27T12:40:28.447682+0000","last_fullsized":"2023-06-27T12:40:28.447682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":413,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.428957+0000","last_change":"2023-06-27T12:40:33.436804+0000","last_active":"2023-06-27T12:40:33.428957+0000","last_peered":"2023-06-27T12:40:33.428957+0000","last_clean":"2023-06-27T12:40:33.428957+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:33.428957+0000","last_undegraded":"2023-06-27T12:40:33.428957+0000","last_fullsized":"2023-06-27T12:40:33.428957+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":355,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.455948+0000","last_change":"2023-06-27T12:40:27.810261+0000","last_active":"2023-06-27T12:40:28.455948+0000","last_peered":"2023-06-27T12:40:28.455948+0000","last_clean":"2023-06-27T12:40:28.455948+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:28.455948+0000","last_undegraded":"2023-06-27T12:40:28.455948+0000","last_fullsized":"2023-06-27T12:40:28.455948+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":255,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.428088+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:33.428088+0000","last_peered":"2023-06-27T12:40:33.428088+0000","last_clean":"2023-06-27T12:40:33.428088+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:33.428088+0000","last_undegraded":"2023-06-27T12:40:33.428088+0000","last_fullsized":"2023-06-27T12:40:33.428088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":247,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.444346+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:33.444346+0000","last_peered":"2023-06-27T12:40:33.444346+0000","last_clean":"2023-06-27T12:40:33.444346+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:33.444346+0000","last_undegraded":"2023-06-27T12:40:33.444346+0000","last_fullsized":"2023-06-27T12:40:33.444346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":411,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-06-27T12:40:33.431181+0000","last_change":"2023-06-27T12:40:33.436080+0000","last_active":"2023-06-27T12:40:33.431181+0000","last_peered":"2023-06-27T12:40:33.431181+0000","last_clean":"2023-06-27T12:40:33.431181+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:33.431181+0000","last_undegraded":"2023-06-27T12:40:33.431181+0000","last_fullsized":"2023-06-27T12:40:33.431181+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":42938,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":104590687,"num_objects":196,"num_object_clones":140,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":876,"num_read_kb":624374,"num_write":965,"num_write_kb":279043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673120,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160440,"kb_used_data":129520,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211400,"statfs":{"total":96636764160,"available":96472473600,"internally_reserved":0,"allocated":132628480,"data_stored":154633646,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705825,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167760,"kb_used_data":136840,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204080,"statfs":{"total":96636764160,"available":96464977920,"internally_reserved":0,"allocated":140124160,"data_stored":162201537,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673120,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":182272,"kb_used_data":151352,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94189568,"statfs":{"total":96636764160,"available":96450117632,"internally_reserved":0,"allocated":154984448,"data_stored":182944933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:40.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~6s 2023-06-27T12:40:42.489 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:42.490 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:42.490 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:42.631 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:42.624+0000 7f81d9b1e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:42.630+0000 7f81d9b1e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:42.827 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:42.829 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:42.840 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2023-06-27T12:40:41.937929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105181055,"num_objects":198,"num_object_clones":140,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":979,"num_read_kb":624465,"num_write":1102,"num_write_kb":280937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":691788,"kb_used_data":568108,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376795572,"statfs":{"total":386547056640,"available":385838665728,"internally_reserved":0,"allocated":581742592,"data_stored":676852329,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.523054"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":323,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583467+0000","last_change":"2023-06-27T12:40:33.819159+0000","last_active":"2023-06-27T12:40:36.583467+0000","last_peered":"2023-06-27T12:40:36.583467+0000","last_clean":"2023-06-27T12:40:36.583467+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:36.583467+0000","last_undegraded":"2023-06-27T12:40:36.583467+0000","last_fullsized":"2023-06-27T12:40:36.583467+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":439,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.442065+0000","last_change":"2023-06-27T12:40:33.439175+0000","last_active":"2023-06-27T12:40:36.442065+0000","last_peered":"2023-06-27T12:40:36.442065+0000","last_clean":"2023-06-27T12:40:36.442065+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:36.442065+0000","last_undegraded":"2023-06-27T12:40:36.442065+0000","last_fullsized":"2023-06-27T12:40:36.442065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":444,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.577791+0000","last_change":"2023-06-27T12:40:33.461948+0000","last_active":"2023-06-27T12:40:36.577791+0000","last_peered":"2023-06-27T12:40:36.577791+0000","last_clean":"2023-06-27T12:40:36.577791+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:36.577791+0000","last_undegraded":"2023-06-27T12:40:36.577791+0000","last_fullsized":"2023-06-27T12:40:36.577791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'67","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591295+0000","last_change":"2023-06-27T12:40:33.824008+0000","last_active":"2023-06-27T12:40:36.591295+0000","last_peered":"2023-06-27T12:40:36.591295+0000","last_clean":"2023-06-27T12:40:36.591295+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:36.591295+0000","last_undegraded":"2023-06-27T12:40:36.591295+0000","last_fullsized":"2023-06-27T12:40:36.591295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050553,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":27199,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":348,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.454098+0000","last_change":"2023-06-27T12:40:27.808791+0000","last_active":"2023-06-27T12:40:28.454098+0000","last_peered":"2023-06-27T12:40:28.454098+0000","last_clean":"2023-06-27T12:40:28.454098+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:28.454098+0000","last_undegraded":"2023-06-27T12:40:28.454098+0000","last_fullsized":"2023-06-27T12:40:28.454098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":497,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583253+0000","last_change":"2023-06-27T12:40:33.818274+0000","last_active":"2023-06-27T12:40:36.583253+0000","last_peered":"2023-06-27T12:40:36.583253+0000","last_clean":"2023-06-27T12:40:36.583253+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:36.583253+0000","last_undegraded":"2023-06-27T12:40:36.583253+0000","last_fullsized":"2023-06-27T12:40:36.583253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":113812,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"163'105","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.594191+0000","last_change":"2023-06-27T12:40:33.883382+0000","last_active":"2023-06-27T12:40:36.594191+0000","last_peered":"2023-06-27T12:40:36.594191+0000","last_clean":"2023-06-27T12:40:36.594191+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:36.594191+0000","last_undegraded":"2023-06-27T12:40:36.594191+0000","last_fullsized":"2023-06-27T12:40:36.594191+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227043,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":57,"num_read_kb":30118,"num_write":63,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":370,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.581982+0000","last_change":"2023-06-27T12:40:33.451340+0000","last_active":"2023-06-27T12:40:36.581982+0000","last_peered":"2023-06-27T12:40:36.581982+0000","last_clean":"2023-06-27T12:40:36.581982+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:36.581982+0000","last_undegraded":"2023-06-27T12:40:36.581982+0000","last_fullsized":"2023-06-27T12:40:36.581982+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2773188,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.438400+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:36.438400+0000","last_peered":"2023-06-27T12:40:36.438400+0000","last_clean":"2023-06-27T12:40:36.438400+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:36.438400+0000","last_undegraded":"2023-06-27T12:40:36.438400+0000","last_fullsized":"2023-06-27T12:40:36.438400+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":318,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.582504+0000","last_change":"2023-06-27T12:40:33.461555+0000","last_active":"2023-06-27T12:40:36.582504+0000","last_peered":"2023-06-27T12:40:36.582504+0000","last_clean":"2023-06-27T12:40:36.582504+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:36.582504+0000","last_undegraded":"2023-06-27T12:40:36.582504+0000","last_fullsized":"2023-06-27T12:40:36.582504+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591210+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:36.591210+0000","last_peered":"2023-06-27T12:40:36.591210+0000","last_clean":"2023-06-27T12:40:36.591210+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:36.591210+0000","last_undegraded":"2023-06-27T12:40:36.591210+0000","last_fullsized":"2023-06-27T12:40:36.591210+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"158'76","reported_seq":395,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.443043+0000","last_change":"2023-06-27T12:40:33.438763+0000","last_active":"2023-06-27T12:40:36.443043+0000","last_peered":"2023-06-27T12:40:36.443043+0000","last_clean":"2023-06-27T12:40:36.443043+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:36.443043+0000","last_undegraded":"2023-06-27T12:40:36.443043+0000","last_fullsized":"2023-06-27T12:40:36.443043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689285,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":36,"num_read_kb":31324,"num_write":50,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.587199+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:36.587199+0000","last_peered":"2023-06-27T12:40:36.587199+0000","last_clean":"2023-06-27T12:40:36.587199+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:36.587199+0000","last_undegraded":"2023-06-27T12:40:36.587199+0000","last_fullsized":"2023-06-27T12:40:36.587199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"160'233","reported_seq":486,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.533100+0000","last_change":"2023-06-27T12:40:28.538999+0000","last_active":"2023-06-27T12:40:28.533100+0000","last_peered":"2023-06-27T12:40:28.533100+0000","last_clean":"2023-06-27T12:40:28.533100+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:28.533100+0000","last_undegraded":"2023-06-27T12:40:28.533100+0000","last_fullsized":"2023-06-27T12:40:28.533100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10319597,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":71887,"num_write":132,"num_write_kb":39413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.575764+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:36.575764+0000","last_peered":"2023-06-27T12:40:36.575764+0000","last_clean":"2023-06-27T12:40:36.575764+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:36.575764+0000","last_undegraded":"2023-06-27T12:40:36.575764+0000","last_fullsized":"2023-06-27T12:40:36.575764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":372,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.449489+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:28.449489+0000","last_peered":"2023-06-27T12:40:28.449489+0000","last_clean":"2023-06-27T12:40:28.449489+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:28.449489+0000","last_undegraded":"2023-06-27T12:40:28.449489+0000","last_fullsized":"2023-06-27T12:40:28.449489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"164'175","reported_seq":504,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.578849+0000","last_change":"2023-06-27T12:40:33.450942+0000","last_active":"2023-06-27T12:40:36.578849+0000","last_peered":"2023-06-27T12:40:36.578849+0000","last_clean":"2023-06-27T12:40:36.578849+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:36.578849+0000","last_undegraded":"2023-06-27T12:40:36.578849+0000","last_fullsized":"2023-06-27T12:40:36.578849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15025524,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":70272,"num_write":126,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.585760+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:36.585760+0000","last_peered":"2023-06-27T12:40:36.585760+0000","last_clean":"2023-06-27T12:40:36.585760+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:36.585760+0000","last_undegraded":"2023-06-27T12:40:36.585760+0000","last_fullsized":"2023-06-27T12:40:36.585760+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"151'73","reported_seq":376,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.446866+0000","last_change":"2023-06-27T12:40:27.809749+0000","last_active":"2023-06-27T12:40:28.446866+0000","last_peered":"2023-06-27T12:40:28.446866+0000","last_clean":"2023-06-27T12:40:28.446866+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:28.446866+0000","last_undegraded":"2023-06-27T12:40:28.446866+0000","last_fullsized":"2023-06-27T12:40:28.446866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840548,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":33,"num_read_kb":24675,"num_write":53,"num_write_kb":15374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.447682+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:28.447682+0000","last_peered":"2023-06-27T12:40:28.447682+0000","last_clean":"2023-06-27T12:40:28.447682+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:28.447682+0000","last_undegraded":"2023-06-27T12:40:28.447682+0000","last_fullsized":"2023-06-27T12:40:28.447682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":417,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.448935+0000","last_change":"2023-06-27T12:40:33.436804+0000","last_active":"2023-06-27T12:40:36.448935+0000","last_peered":"2023-06-27T12:40:36.448935+0000","last_clean":"2023-06-27T12:40:36.448935+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:36.448935+0000","last_undegraded":"2023-06-27T12:40:36.448935+0000","last_fullsized":"2023-06-27T12:40:36.448935+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":355,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.455948+0000","last_change":"2023-06-27T12:40:27.810261+0000","last_active":"2023-06-27T12:40:28.455948+0000","last_peered":"2023-06-27T12:40:28.455948+0000","last_clean":"2023-06-27T12:40:28.455948+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:28.455948+0000","last_undegraded":"2023-06-27T12:40:28.455948+0000","last_fullsized":"2023-06-27T12:40:28.455948+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.449460+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:36.449460+0000","last_peered":"2023-06-27T12:40:36.449460+0000","last_clean":"2023-06-27T12:40:36.449460+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:36.449460+0000","last_undegraded":"2023-06-27T12:40:36.449460+0000","last_fullsized":"2023-06-27T12:40:36.449460+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591666+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:36.591666+0000","last_peered":"2023-06-27T12:40:36.591666+0000","last_clean":"2023-06-27T12:40:36.591666+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:36.591666+0000","last_undegraded":"2023-06-27T12:40:36.591666+0000","last_fullsized":"2023-06-27T12:40:36.591666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":415,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.436802+0000","last_change":"2023-06-27T12:40:33.436080+0000","last_active":"2023-06-27T12:40:36.436802+0000","last_peered":"2023-06-27T12:40:36.436802+0000","last_clean":"2023-06-27T12:40:36.436802+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:36.436802+0000","last_undegraded":"2023-06-27T12:40:36.436802+0000","last_fullsized":"2023-06-27T12:40:36.436802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":42938,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":104590687,"num_objects":196,"num_object_clones":140,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":876,"num_read_kb":624374,"num_write":965,"num_write_kb":279043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673125,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160452,"kb_used_data":129532,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211388,"statfs":{"total":96636764160,"available":96472461312,"internally_reserved":0,"allocated":132640768,"data_stored":154644533,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705830,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167772,"kb_used_data":136852,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204068,"statfs":{"total":96636764160,"available":96464965632,"internally_reserved":0,"allocated":140136448,"data_stored":162212424,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673125,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":182284,"kb_used_data":151364,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94189556,"statfs":{"total":96636764160,"available":96450105344,"internally_reserved":0,"allocated":154996736,"data_stored":182955820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:46.312 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~11s 2023-06-27T12:40:47.843 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:47.843 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:47.844 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:47.985 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:47.978+0000 7f0cf073b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:47.988 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:47.985+0000 7f0cf073b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:48.182 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:48.184 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:48.196 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2023-06-27T12:40:47.939222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105181055,"num_objects":198,"num_object_clones":140,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":979,"num_read_kb":624465,"num_write":1102,"num_write_kb":280937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":691788,"kb_used_data":568108,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376795572,"statfs":{"total":386547056640,"available":385838665728,"internally_reserved":0,"allocated":581742592,"data_stored":676852329,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.516212"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":323,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583467+0000","last_change":"2023-06-27T12:40:33.819159+0000","last_active":"2023-06-27T12:40:36.583467+0000","last_peered":"2023-06-27T12:40:36.583467+0000","last_clean":"2023-06-27T12:40:36.583467+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:36.583467+0000","last_undegraded":"2023-06-27T12:40:36.583467+0000","last_fullsized":"2023-06-27T12:40:36.583467+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":439,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.442065+0000","last_change":"2023-06-27T12:40:33.439175+0000","last_active":"2023-06-27T12:40:36.442065+0000","last_peered":"2023-06-27T12:40:36.442065+0000","last_clean":"2023-06-27T12:40:36.442065+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:36.442065+0000","last_undegraded":"2023-06-27T12:40:36.442065+0000","last_fullsized":"2023-06-27T12:40:36.442065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":444,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.577791+0000","last_change":"2023-06-27T12:40:33.461948+0000","last_active":"2023-06-27T12:40:36.577791+0000","last_peered":"2023-06-27T12:40:36.577791+0000","last_clean":"2023-06-27T12:40:36.577791+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:36.577791+0000","last_undegraded":"2023-06-27T12:40:36.577791+0000","last_fullsized":"2023-06-27T12:40:36.577791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'67","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591295+0000","last_change":"2023-06-27T12:40:33.824008+0000","last_active":"2023-06-27T12:40:36.591295+0000","last_peered":"2023-06-27T12:40:36.591295+0000","last_clean":"2023-06-27T12:40:36.591295+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:36.591295+0000","last_undegraded":"2023-06-27T12:40:36.591295+0000","last_fullsized":"2023-06-27T12:40:36.591295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050553,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":27199,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":348,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.454098+0000","last_change":"2023-06-27T12:40:27.808791+0000","last_active":"2023-06-27T12:40:28.454098+0000","last_peered":"2023-06-27T12:40:28.454098+0000","last_clean":"2023-06-27T12:40:28.454098+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:28.454098+0000","last_undegraded":"2023-06-27T12:40:28.454098+0000","last_fullsized":"2023-06-27T12:40:28.454098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":497,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583253+0000","last_change":"2023-06-27T12:40:33.818274+0000","last_active":"2023-06-27T12:40:36.583253+0000","last_peered":"2023-06-27T12:40:36.583253+0000","last_clean":"2023-06-27T12:40:36.583253+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:36.583253+0000","last_undegraded":"2023-06-27T12:40:36.583253+0000","last_fullsized":"2023-06-27T12:40:36.583253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":113812,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"163'105","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.594191+0000","last_change":"2023-06-27T12:40:33.883382+0000","last_active":"2023-06-27T12:40:36.594191+0000","last_peered":"2023-06-27T12:40:36.594191+0000","last_clean":"2023-06-27T12:40:36.594191+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:36.594191+0000","last_undegraded":"2023-06-27T12:40:36.594191+0000","last_fullsized":"2023-06-27T12:40:36.594191+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227043,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":57,"num_read_kb":30118,"num_write":63,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":370,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.581982+0000","last_change":"2023-06-27T12:40:33.451340+0000","last_active":"2023-06-27T12:40:36.581982+0000","last_peered":"2023-06-27T12:40:36.581982+0000","last_clean":"2023-06-27T12:40:36.581982+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:36.581982+0000","last_undegraded":"2023-06-27T12:40:36.581982+0000","last_fullsized":"2023-06-27T12:40:36.581982+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2773188,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.438400+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:36.438400+0000","last_peered":"2023-06-27T12:40:36.438400+0000","last_clean":"2023-06-27T12:40:36.438400+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:36.438400+0000","last_undegraded":"2023-06-27T12:40:36.438400+0000","last_fullsized":"2023-06-27T12:40:36.438400+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":318,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.582504+0000","last_change":"2023-06-27T12:40:33.461555+0000","last_active":"2023-06-27T12:40:36.582504+0000","last_peered":"2023-06-27T12:40:36.582504+0000","last_clean":"2023-06-27T12:40:36.582504+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:36.582504+0000","last_undegraded":"2023-06-27T12:40:36.582504+0000","last_fullsized":"2023-06-27T12:40:36.582504+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591210+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:36.591210+0000","last_peered":"2023-06-27T12:40:36.591210+0000","last_clean":"2023-06-27T12:40:36.591210+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:36.591210+0000","last_undegraded":"2023-06-27T12:40:36.591210+0000","last_fullsized":"2023-06-27T12:40:36.591210+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"158'76","reported_seq":395,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.443043+0000","last_change":"2023-06-27T12:40:33.438763+0000","last_active":"2023-06-27T12:40:36.443043+0000","last_peered":"2023-06-27T12:40:36.443043+0000","last_clean":"2023-06-27T12:40:36.443043+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:36.443043+0000","last_undegraded":"2023-06-27T12:40:36.443043+0000","last_fullsized":"2023-06-27T12:40:36.443043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689285,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":36,"num_read_kb":31324,"num_write":50,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.587199+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:36.587199+0000","last_peered":"2023-06-27T12:40:36.587199+0000","last_clean":"2023-06-27T12:40:36.587199+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:36.587199+0000","last_undegraded":"2023-06-27T12:40:36.587199+0000","last_fullsized":"2023-06-27T12:40:36.587199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"160'233","reported_seq":486,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.533100+0000","last_change":"2023-06-27T12:40:28.538999+0000","last_active":"2023-06-27T12:40:28.533100+0000","last_peered":"2023-06-27T12:40:28.533100+0000","last_clean":"2023-06-27T12:40:28.533100+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:28.533100+0000","last_undegraded":"2023-06-27T12:40:28.533100+0000","last_fullsized":"2023-06-27T12:40:28.533100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10319597,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":71887,"num_write":132,"num_write_kb":39413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.575764+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:36.575764+0000","last_peered":"2023-06-27T12:40:36.575764+0000","last_clean":"2023-06-27T12:40:36.575764+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:36.575764+0000","last_undegraded":"2023-06-27T12:40:36.575764+0000","last_fullsized":"2023-06-27T12:40:36.575764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":372,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.449489+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:28.449489+0000","last_peered":"2023-06-27T12:40:28.449489+0000","last_clean":"2023-06-27T12:40:28.449489+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:28.449489+0000","last_undegraded":"2023-06-27T12:40:28.449489+0000","last_fullsized":"2023-06-27T12:40:28.449489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"164'175","reported_seq":504,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.578849+0000","last_change":"2023-06-27T12:40:33.450942+0000","last_active":"2023-06-27T12:40:36.578849+0000","last_peered":"2023-06-27T12:40:36.578849+0000","last_clean":"2023-06-27T12:40:36.578849+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:36.578849+0000","last_undegraded":"2023-06-27T12:40:36.578849+0000","last_fullsized":"2023-06-27T12:40:36.578849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15025524,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":70272,"num_write":126,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.585760+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:36.585760+0000","last_peered":"2023-06-27T12:40:36.585760+0000","last_clean":"2023-06-27T12:40:36.585760+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:36.585760+0000","last_undegraded":"2023-06-27T12:40:36.585760+0000","last_fullsized":"2023-06-27T12:40:36.585760+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"151'73","reported_seq":376,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.446866+0000","last_change":"2023-06-27T12:40:27.809749+0000","last_active":"2023-06-27T12:40:28.446866+0000","last_peered":"2023-06-27T12:40:28.446866+0000","last_clean":"2023-06-27T12:40:28.446866+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:28.446866+0000","last_undegraded":"2023-06-27T12:40:28.446866+0000","last_fullsized":"2023-06-27T12:40:28.446866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840548,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":33,"num_read_kb":24675,"num_write":53,"num_write_kb":15374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.447682+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:28.447682+0000","last_peered":"2023-06-27T12:40:28.447682+0000","last_clean":"2023-06-27T12:40:28.447682+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:28.447682+0000","last_undegraded":"2023-06-27T12:40:28.447682+0000","last_fullsized":"2023-06-27T12:40:28.447682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":417,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.448935+0000","last_change":"2023-06-27T12:40:33.436804+0000","last_active":"2023-06-27T12:40:36.448935+0000","last_peered":"2023-06-27T12:40:36.448935+0000","last_clean":"2023-06-27T12:40:36.448935+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:36.448935+0000","last_undegraded":"2023-06-27T12:40:36.448935+0000","last_fullsized":"2023-06-27T12:40:36.448935+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":355,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.455948+0000","last_change":"2023-06-27T12:40:27.810261+0000","last_active":"2023-06-27T12:40:28.455948+0000","last_peered":"2023-06-27T12:40:28.455948+0000","last_clean":"2023-06-27T12:40:28.455948+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:28.455948+0000","last_undegraded":"2023-06-27T12:40:28.455948+0000","last_fullsized":"2023-06-27T12:40:28.455948+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.449460+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:36.449460+0000","last_peered":"2023-06-27T12:40:36.449460+0000","last_clean":"2023-06-27T12:40:36.449460+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:36.449460+0000","last_undegraded":"2023-06-27T12:40:36.449460+0000","last_fullsized":"2023-06-27T12:40:36.449460+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591666+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:36.591666+0000","last_peered":"2023-06-27T12:40:36.591666+0000","last_clean":"2023-06-27T12:40:36.591666+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:36.591666+0000","last_undegraded":"2023-06-27T12:40:36.591666+0000","last_fullsized":"2023-06-27T12:40:36.591666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":415,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.436802+0000","last_change":"2023-06-27T12:40:33.436080+0000","last_active":"2023-06-27T12:40:36.436802+0000","last_peered":"2023-06-27T12:40:36.436802+0000","last_clean":"2023-06-27T12:40:36.436802+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:36.436802+0000","last_undegraded":"2023-06-27T12:40:36.436802+0000","last_fullsized":"2023-06-27T12:40:36.436802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":42938,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":104590687,"num_objects":196,"num_object_clones":140,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":876,"num_read_kb":624374,"num_write":965,"num_write_kb":279043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673130,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160452,"kb_used_data":129532,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211388,"statfs":{"total":96636764160,"available":96472461312,"internally_reserved":0,"allocated":132640768,"data_stored":154644533,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705835,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167772,"kb_used_data":136852,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204068,"statfs":{"total":96636764160,"available":96464965632,"internally_reserved":0,"allocated":140136448,"data_stored":162212424,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673130,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":182284,"kb_used_data":151364,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94189556,"statfs":{"total":96636764160,"available":96450105344,"internally_reserved":0,"allocated":154996736,"data_stored":182955820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:51.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~17s 2023-06-27T12:40:53.198 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:53.198 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:53.199 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:53.341 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:53.334+0000 7f9b3d896700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:53.344 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:53.341+0000 7f9b3d896700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:53.540 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:53.552 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2023-06-27T12:40:51.939993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105181055,"num_objects":198,"num_object_clones":140,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":979,"num_read_kb":624465,"num_write":1102,"num_write_kb":280937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":691788,"kb_used_data":568108,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376795572,"statfs":{"total":386547056640,"available":385838665728,"internally_reserved":0,"allocated":581742592,"data_stored":676852329,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002613"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":323,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583467+0000","last_change":"2023-06-27T12:40:33.819159+0000","last_active":"2023-06-27T12:40:36.583467+0000","last_peered":"2023-06-27T12:40:36.583467+0000","last_clean":"2023-06-27T12:40:36.583467+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:36.583467+0000","last_undegraded":"2023-06-27T12:40:36.583467+0000","last_fullsized":"2023-06-27T12:40:36.583467+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":439,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.442065+0000","last_change":"2023-06-27T12:40:33.439175+0000","last_active":"2023-06-27T12:40:36.442065+0000","last_peered":"2023-06-27T12:40:36.442065+0000","last_clean":"2023-06-27T12:40:36.442065+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:36.442065+0000","last_undegraded":"2023-06-27T12:40:36.442065+0000","last_fullsized":"2023-06-27T12:40:36.442065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":444,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.577791+0000","last_change":"2023-06-27T12:40:33.461948+0000","last_active":"2023-06-27T12:40:36.577791+0000","last_peered":"2023-06-27T12:40:36.577791+0000","last_clean":"2023-06-27T12:40:36.577791+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:36.577791+0000","last_undegraded":"2023-06-27T12:40:36.577791+0000","last_fullsized":"2023-06-27T12:40:36.577791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'67","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591295+0000","last_change":"2023-06-27T12:40:33.824008+0000","last_active":"2023-06-27T12:40:36.591295+0000","last_peered":"2023-06-27T12:40:36.591295+0000","last_clean":"2023-06-27T12:40:36.591295+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:36.591295+0000","last_undegraded":"2023-06-27T12:40:36.591295+0000","last_fullsized":"2023-06-27T12:40:36.591295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050553,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":27199,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":348,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.454098+0000","last_change":"2023-06-27T12:40:27.808791+0000","last_active":"2023-06-27T12:40:28.454098+0000","last_peered":"2023-06-27T12:40:28.454098+0000","last_clean":"2023-06-27T12:40:28.454098+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:28.454098+0000","last_undegraded":"2023-06-27T12:40:28.454098+0000","last_fullsized":"2023-06-27T12:40:28.454098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":497,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583253+0000","last_change":"2023-06-27T12:40:33.818274+0000","last_active":"2023-06-27T12:40:36.583253+0000","last_peered":"2023-06-27T12:40:36.583253+0000","last_clean":"2023-06-27T12:40:36.583253+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:36.583253+0000","last_undegraded":"2023-06-27T12:40:36.583253+0000","last_fullsized":"2023-06-27T12:40:36.583253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":113812,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"163'105","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.594191+0000","last_change":"2023-06-27T12:40:33.883382+0000","last_active":"2023-06-27T12:40:36.594191+0000","last_peered":"2023-06-27T12:40:36.594191+0000","last_clean":"2023-06-27T12:40:36.594191+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:36.594191+0000","last_undegraded":"2023-06-27T12:40:36.594191+0000","last_fullsized":"2023-06-27T12:40:36.594191+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227043,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":57,"num_read_kb":30118,"num_write":63,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":370,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.581982+0000","last_change":"2023-06-27T12:40:33.451340+0000","last_active":"2023-06-27T12:40:36.581982+0000","last_peered":"2023-06-27T12:40:36.581982+0000","last_clean":"2023-06-27T12:40:36.581982+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:36.581982+0000","last_undegraded":"2023-06-27T12:40:36.581982+0000","last_fullsized":"2023-06-27T12:40:36.581982+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2773188,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.438400+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:36.438400+0000","last_peered":"2023-06-27T12:40:36.438400+0000","last_clean":"2023-06-27T12:40:36.438400+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:36.438400+0000","last_undegraded":"2023-06-27T12:40:36.438400+0000","last_fullsized":"2023-06-27T12:40:36.438400+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":318,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.582504+0000","last_change":"2023-06-27T12:40:33.461555+0000","last_active":"2023-06-27T12:40:36.582504+0000","last_peered":"2023-06-27T12:40:36.582504+0000","last_clean":"2023-06-27T12:40:36.582504+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:36.582504+0000","last_undegraded":"2023-06-27T12:40:36.582504+0000","last_fullsized":"2023-06-27T12:40:36.582504+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591210+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:36.591210+0000","last_peered":"2023-06-27T12:40:36.591210+0000","last_clean":"2023-06-27T12:40:36.591210+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:36.591210+0000","last_undegraded":"2023-06-27T12:40:36.591210+0000","last_fullsized":"2023-06-27T12:40:36.591210+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"158'76","reported_seq":395,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.443043+0000","last_change":"2023-06-27T12:40:33.438763+0000","last_active":"2023-06-27T12:40:36.443043+0000","last_peered":"2023-06-27T12:40:36.443043+0000","last_clean":"2023-06-27T12:40:36.443043+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:36.443043+0000","last_undegraded":"2023-06-27T12:40:36.443043+0000","last_fullsized":"2023-06-27T12:40:36.443043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689285,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":36,"num_read_kb":31324,"num_write":50,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.587199+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:36.587199+0000","last_peered":"2023-06-27T12:40:36.587199+0000","last_clean":"2023-06-27T12:40:36.587199+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:36.587199+0000","last_undegraded":"2023-06-27T12:40:36.587199+0000","last_fullsized":"2023-06-27T12:40:36.587199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"160'233","reported_seq":486,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.533100+0000","last_change":"2023-06-27T12:40:28.538999+0000","last_active":"2023-06-27T12:40:28.533100+0000","last_peered":"2023-06-27T12:40:28.533100+0000","last_clean":"2023-06-27T12:40:28.533100+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:28.533100+0000","last_undegraded":"2023-06-27T12:40:28.533100+0000","last_fullsized":"2023-06-27T12:40:28.533100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10319597,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":71887,"num_write":132,"num_write_kb":39413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.575764+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:36.575764+0000","last_peered":"2023-06-27T12:40:36.575764+0000","last_clean":"2023-06-27T12:40:36.575764+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:36.575764+0000","last_undegraded":"2023-06-27T12:40:36.575764+0000","last_fullsized":"2023-06-27T12:40:36.575764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":372,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.449489+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:28.449489+0000","last_peered":"2023-06-27T12:40:28.449489+0000","last_clean":"2023-06-27T12:40:28.449489+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:28.449489+0000","last_undegraded":"2023-06-27T12:40:28.449489+0000","last_fullsized":"2023-06-27T12:40:28.449489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"164'175","reported_seq":504,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.578849+0000","last_change":"2023-06-27T12:40:33.450942+0000","last_active":"2023-06-27T12:40:36.578849+0000","last_peered":"2023-06-27T12:40:36.578849+0000","last_clean":"2023-06-27T12:40:36.578849+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:36.578849+0000","last_undegraded":"2023-06-27T12:40:36.578849+0000","last_fullsized":"2023-06-27T12:40:36.578849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15025524,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":70272,"num_write":126,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.585760+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:36.585760+0000","last_peered":"2023-06-27T12:40:36.585760+0000","last_clean":"2023-06-27T12:40:36.585760+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:36.585760+0000","last_undegraded":"2023-06-27T12:40:36.585760+0000","last_fullsized":"2023-06-27T12:40:36.585760+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"151'73","reported_seq":376,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.446866+0000","last_change":"2023-06-27T12:40:27.809749+0000","last_active":"2023-06-27T12:40:28.446866+0000","last_peered":"2023-06-27T12:40:28.446866+0000","last_clean":"2023-06-27T12:40:28.446866+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:28.446866+0000","last_undegraded":"2023-06-27T12:40:28.446866+0000","last_fullsized":"2023-06-27T12:40:28.446866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840548,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":33,"num_read_kb":24675,"num_write":53,"num_write_kb":15374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.447682+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:28.447682+0000","last_peered":"2023-06-27T12:40:28.447682+0000","last_clean":"2023-06-27T12:40:28.447682+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:28.447682+0000","last_undegraded":"2023-06-27T12:40:28.447682+0000","last_fullsized":"2023-06-27T12:40:28.447682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":417,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.448935+0000","last_change":"2023-06-27T12:40:33.436804+0000","last_active":"2023-06-27T12:40:36.448935+0000","last_peered":"2023-06-27T12:40:36.448935+0000","last_clean":"2023-06-27T12:40:36.448935+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:36.448935+0000","last_undegraded":"2023-06-27T12:40:36.448935+0000","last_fullsized":"2023-06-27T12:40:36.448935+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":355,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.455948+0000","last_change":"2023-06-27T12:40:27.810261+0000","last_active":"2023-06-27T12:40:28.455948+0000","last_peered":"2023-06-27T12:40:28.455948+0000","last_clean":"2023-06-27T12:40:28.455948+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:28.455948+0000","last_undegraded":"2023-06-27T12:40:28.455948+0000","last_fullsized":"2023-06-27T12:40:28.455948+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.449460+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:36.449460+0000","last_peered":"2023-06-27T12:40:36.449460+0000","last_clean":"2023-06-27T12:40:36.449460+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:36.449460+0000","last_undegraded":"2023-06-27T12:40:36.449460+0000","last_fullsized":"2023-06-27T12:40:36.449460+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591666+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:36.591666+0000","last_peered":"2023-06-27T12:40:36.591666+0000","last_clean":"2023-06-27T12:40:36.591666+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:36.591666+0000","last_undegraded":"2023-06-27T12:40:36.591666+0000","last_fullsized":"2023-06-27T12:40:36.591666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":415,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.436802+0000","last_change":"2023-06-27T12:40:33.436080+0000","last_active":"2023-06-27T12:40:36.436802+0000","last_peered":"2023-06-27T12:40:36.436802+0000","last_clean":"2023-06-27T12:40:36.436802+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:36.436802+0000","last_undegraded":"2023-06-27T12:40:36.436802+0000","last_fullsized":"2023-06-27T12:40:36.436802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":42938,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":104590687,"num_objects":196,"num_object_clones":140,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":876,"num_read_kb":624374,"num_write":965,"num_write_kb":279043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673135,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160452,"kb_used_data":129532,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211388,"statfs":{"total":96636764160,"available":96472461312,"internally_reserved":0,"allocated":132640768,"data_stored":154644533,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705840,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167772,"kb_used_data":136852,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204068,"statfs":{"total":96636764160,"available":96464965632,"internally_reserved":0,"allocated":140136448,"data_stored":162212424,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673135,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":182284,"kb_used_data":151364,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94189556,"statfs":{"total":96636764160,"available":96450105344,"internally_reserved":0,"allocated":154996736,"data_stored":182955820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:57.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~22s 2023-06-27T12:40:58.554 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:40:58.554 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:40:58.555 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:40:58.696 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:58.689+0000 7fd0a40be700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:58.699 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:40:58.696+0000 7fd0a40be700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:40:58.893 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:40:58.894 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:40:58.906 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2023-06-27T12:40:57.941215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105181055,"num_objects":198,"num_object_clones":140,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":979,"num_read_kb":624465,"num_write":1102,"num_write_kb":280937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":691788,"kb_used_data":568108,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376795572,"statfs":{"total":386547056640,"available":385838665728,"internally_reserved":0,"allocated":581742592,"data_stored":676852329,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002439"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":323,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583467+0000","last_change":"2023-06-27T12:40:33.819159+0000","last_active":"2023-06-27T12:40:36.583467+0000","last_peered":"2023-06-27T12:40:36.583467+0000","last_clean":"2023-06-27T12:40:36.583467+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:36.583467+0000","last_undegraded":"2023-06-27T12:40:36.583467+0000","last_fullsized":"2023-06-27T12:40:36.583467+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":439,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.442065+0000","last_change":"2023-06-27T12:40:33.439175+0000","last_active":"2023-06-27T12:40:36.442065+0000","last_peered":"2023-06-27T12:40:36.442065+0000","last_clean":"2023-06-27T12:40:36.442065+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:36.442065+0000","last_undegraded":"2023-06-27T12:40:36.442065+0000","last_fullsized":"2023-06-27T12:40:36.442065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":444,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.577791+0000","last_change":"2023-06-27T12:40:33.461948+0000","last_active":"2023-06-27T12:40:36.577791+0000","last_peered":"2023-06-27T12:40:36.577791+0000","last_clean":"2023-06-27T12:40:36.577791+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:36.577791+0000","last_undegraded":"2023-06-27T12:40:36.577791+0000","last_fullsized":"2023-06-27T12:40:36.577791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'67","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591295+0000","last_change":"2023-06-27T12:40:33.824008+0000","last_active":"2023-06-27T12:40:36.591295+0000","last_peered":"2023-06-27T12:40:36.591295+0000","last_clean":"2023-06-27T12:40:36.591295+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:36.591295+0000","last_undegraded":"2023-06-27T12:40:36.591295+0000","last_fullsized":"2023-06-27T12:40:36.591295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050553,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":27199,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":348,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.454098+0000","last_change":"2023-06-27T12:40:27.808791+0000","last_active":"2023-06-27T12:40:28.454098+0000","last_peered":"2023-06-27T12:40:28.454098+0000","last_clean":"2023-06-27T12:40:28.454098+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:28.454098+0000","last_undegraded":"2023-06-27T12:40:28.454098+0000","last_fullsized":"2023-06-27T12:40:28.454098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":497,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583253+0000","last_change":"2023-06-27T12:40:33.818274+0000","last_active":"2023-06-27T12:40:36.583253+0000","last_peered":"2023-06-27T12:40:36.583253+0000","last_clean":"2023-06-27T12:40:36.583253+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:36.583253+0000","last_undegraded":"2023-06-27T12:40:36.583253+0000","last_fullsized":"2023-06-27T12:40:36.583253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":113812,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"163'105","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.594191+0000","last_change":"2023-06-27T12:40:33.883382+0000","last_active":"2023-06-27T12:40:36.594191+0000","last_peered":"2023-06-27T12:40:36.594191+0000","last_clean":"2023-06-27T12:40:36.594191+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:36.594191+0000","last_undegraded":"2023-06-27T12:40:36.594191+0000","last_fullsized":"2023-06-27T12:40:36.594191+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227043,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":57,"num_read_kb":30118,"num_write":63,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":370,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.581982+0000","last_change":"2023-06-27T12:40:33.451340+0000","last_active":"2023-06-27T12:40:36.581982+0000","last_peered":"2023-06-27T12:40:36.581982+0000","last_clean":"2023-06-27T12:40:36.581982+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:36.581982+0000","last_undegraded":"2023-06-27T12:40:36.581982+0000","last_fullsized":"2023-06-27T12:40:36.581982+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2773188,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.438400+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:36.438400+0000","last_peered":"2023-06-27T12:40:36.438400+0000","last_clean":"2023-06-27T12:40:36.438400+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:36.438400+0000","last_undegraded":"2023-06-27T12:40:36.438400+0000","last_fullsized":"2023-06-27T12:40:36.438400+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":318,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.582504+0000","last_change":"2023-06-27T12:40:33.461555+0000","last_active":"2023-06-27T12:40:36.582504+0000","last_peered":"2023-06-27T12:40:36.582504+0000","last_clean":"2023-06-27T12:40:36.582504+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:36.582504+0000","last_undegraded":"2023-06-27T12:40:36.582504+0000","last_fullsized":"2023-06-27T12:40:36.582504+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591210+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:36.591210+0000","last_peered":"2023-06-27T12:40:36.591210+0000","last_clean":"2023-06-27T12:40:36.591210+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:36.591210+0000","last_undegraded":"2023-06-27T12:40:36.591210+0000","last_fullsized":"2023-06-27T12:40:36.591210+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"158'76","reported_seq":395,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.443043+0000","last_change":"2023-06-27T12:40:33.438763+0000","last_active":"2023-06-27T12:40:36.443043+0000","last_peered":"2023-06-27T12:40:36.443043+0000","last_clean":"2023-06-27T12:40:36.443043+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:36.443043+0000","last_undegraded":"2023-06-27T12:40:36.443043+0000","last_fullsized":"2023-06-27T12:40:36.443043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689285,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":36,"num_read_kb":31324,"num_write":50,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.587199+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:36.587199+0000","last_peered":"2023-06-27T12:40:36.587199+0000","last_clean":"2023-06-27T12:40:36.587199+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:36.587199+0000","last_undegraded":"2023-06-27T12:40:36.587199+0000","last_fullsized":"2023-06-27T12:40:36.587199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"160'233","reported_seq":486,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.533100+0000","last_change":"2023-06-27T12:40:28.538999+0000","last_active":"2023-06-27T12:40:28.533100+0000","last_peered":"2023-06-27T12:40:28.533100+0000","last_clean":"2023-06-27T12:40:28.533100+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:28.533100+0000","last_undegraded":"2023-06-27T12:40:28.533100+0000","last_fullsized":"2023-06-27T12:40:28.533100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10319597,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":71887,"num_write":132,"num_write_kb":39413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.575764+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:36.575764+0000","last_peered":"2023-06-27T12:40:36.575764+0000","last_clean":"2023-06-27T12:40:36.575764+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:36.575764+0000","last_undegraded":"2023-06-27T12:40:36.575764+0000","last_fullsized":"2023-06-27T12:40:36.575764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":372,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.449489+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:28.449489+0000","last_peered":"2023-06-27T12:40:28.449489+0000","last_clean":"2023-06-27T12:40:28.449489+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:28.449489+0000","last_undegraded":"2023-06-27T12:40:28.449489+0000","last_fullsized":"2023-06-27T12:40:28.449489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"164'175","reported_seq":504,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.578849+0000","last_change":"2023-06-27T12:40:33.450942+0000","last_active":"2023-06-27T12:40:36.578849+0000","last_peered":"2023-06-27T12:40:36.578849+0000","last_clean":"2023-06-27T12:40:36.578849+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:36.578849+0000","last_undegraded":"2023-06-27T12:40:36.578849+0000","last_fullsized":"2023-06-27T12:40:36.578849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15025524,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":70272,"num_write":126,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.585760+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:36.585760+0000","last_peered":"2023-06-27T12:40:36.585760+0000","last_clean":"2023-06-27T12:40:36.585760+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:36.585760+0000","last_undegraded":"2023-06-27T12:40:36.585760+0000","last_fullsized":"2023-06-27T12:40:36.585760+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"151'73","reported_seq":376,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.446866+0000","last_change":"2023-06-27T12:40:27.809749+0000","last_active":"2023-06-27T12:40:28.446866+0000","last_peered":"2023-06-27T12:40:28.446866+0000","last_clean":"2023-06-27T12:40:28.446866+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:28.446866+0000","last_undegraded":"2023-06-27T12:40:28.446866+0000","last_fullsized":"2023-06-27T12:40:28.446866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840548,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":33,"num_read_kb":24675,"num_write":53,"num_write_kb":15374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.447682+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:28.447682+0000","last_peered":"2023-06-27T12:40:28.447682+0000","last_clean":"2023-06-27T12:40:28.447682+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:28.447682+0000","last_undegraded":"2023-06-27T12:40:28.447682+0000","last_fullsized":"2023-06-27T12:40:28.447682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":417,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.448935+0000","last_change":"2023-06-27T12:40:33.436804+0000","last_active":"2023-06-27T12:40:36.448935+0000","last_peered":"2023-06-27T12:40:36.448935+0000","last_clean":"2023-06-27T12:40:36.448935+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:36.448935+0000","last_undegraded":"2023-06-27T12:40:36.448935+0000","last_fullsized":"2023-06-27T12:40:36.448935+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":355,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-06-27T12:40:28.455948+0000","last_change":"2023-06-27T12:40:27.810261+0000","last_active":"2023-06-27T12:40:28.455948+0000","last_peered":"2023-06-27T12:40:28.455948+0000","last_clean":"2023-06-27T12:40:28.455948+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:28.455948+0000","last_undegraded":"2023-06-27T12:40:28.455948+0000","last_fullsized":"2023-06-27T12:40:28.455948+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.449460+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:36.449460+0000","last_peered":"2023-06-27T12:40:36.449460+0000","last_clean":"2023-06-27T12:40:36.449460+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:36.449460+0000","last_undegraded":"2023-06-27T12:40:36.449460+0000","last_fullsized":"2023-06-27T12:40:36.449460+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591666+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:36.591666+0000","last_peered":"2023-06-27T12:40:36.591666+0000","last_clean":"2023-06-27T12:40:36.591666+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:36.591666+0000","last_undegraded":"2023-06-27T12:40:36.591666+0000","last_fullsized":"2023-06-27T12:40:36.591666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":415,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.436802+0000","last_change":"2023-06-27T12:40:33.436080+0000","last_active":"2023-06-27T12:40:36.436802+0000","last_peered":"2023-06-27T12:40:36.436802+0000","last_clean":"2023-06-27T12:40:36.436802+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:36.436802+0000","last_undegraded":"2023-06-27T12:40:36.436802+0000","last_fullsized":"2023-06-27T12:40:36.436802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":42938,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":104590687,"num_objects":196,"num_object_clones":140,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":876,"num_read_kb":624374,"num_write":965,"num_write_kb":279043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673140,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160452,"kb_used_data":129532,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211388,"statfs":{"total":96636764160,"available":96472461312,"internally_reserved":0,"allocated":132640768,"data_stored":154644533,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705845,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167772,"kb_used_data":136852,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204068,"statfs":{"total":96636764160,"available":96464965632,"internally_reserved":0,"allocated":140136448,"data_stored":162212424,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673140,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":182284,"kb_used_data":151364,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94189556,"statfs":{"total":96636764160,"available":96450105344,"internally_reserved":0,"allocated":154996736,"data_stored":182955820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:40:59.458 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:40:59.462 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:40:59.462 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:00.539 INFO:tasks.rados.rados.0.smithi138.stdout:592: done (2 left) 2023-06-27T12:41:00.584 INFO:tasks.rados.rados.0.smithi138.stdout:593: write (excl) oid 32 current snap is 78 2023-06-27T12:41:00.584 INFO:tasks.rados.rados.0.smithi138.stdout:593: seq_num 287 ranges {423609=701524,1628332=472089,2648987=174622} 2023-06-27T12:41:00.587 INFO:tasks.rados.rados.0.smithi138.stdout:593: writing smithi138138488-32 from 423609 to 1125133 tid 1 2023-06-27T12:41:00.588 INFO:tasks.rados.rados.0.smithi138.stdout:593: writing smithi138138488-32 from 1628332 to 2100421 tid 2 2023-06-27T12:41:00.588 INFO:tasks.rados.rados.0.smithi138.stdout:593: writing smithi138138488-32 from 2648987 to 2823609 tid 3 2023-06-27T12:41:00.588 INFO:tasks.rados.rados.0.smithi138.stdout:594: read oid 26 snap -1 2023-06-27T12:41:00.589 INFO:tasks.rados.rados.0.smithi138.stdout:594: expect (ObjNum 258 snap 70 seq_num 258) 2023-06-27T12:41:00.589 INFO:tasks.rados.rados.0.smithi138.stdout:595: write oid 30 current snap is 78 2023-06-27T12:41:00.589 INFO:tasks.rados.rados.0.smithi138.stdout:595: seq_num 288 ranges {440416=776773,1686814=732100} 2023-06-27T12:41:00.589 INFO:tasks.rados.rados.0.smithi138.stdout:595: writing smithi138138488-30 from 440416 to 1217189 tid 1 2023-06-27T12:41:00.590 INFO:tasks.rados.rados.0.smithi138.stdout:595: writing smithi138138488-30 from 1686814 to 2418914 tid 2 2023-06-27T12:41:00.590 INFO:tasks.rados.rados.0.smithi138.stdout:596: write (excl) oid 5 current snap is 78 2023-06-27T12:41:00.590 INFO:tasks.rados.rados.0.smithi138.stdout:596: seq_num 289 ranges {457223=568375,1461649=475758,2420186=437037} 2023-06-27T12:41:00.591 INFO:tasks.rados.rados.0.smithi138.stdout:596: writing smithi138138488-5 from 457223 to 1025598 tid 1 2023-06-27T12:41:00.591 INFO:tasks.rados.rados.0.smithi138.stdout:596: writing smithi138138488-5 from 1461649 to 1937407 tid 2 2023-06-27T12:41:00.591 INFO:tasks.rados.rados.0.smithi138.stdout:596: writing smithi138138488-5 from 2420186 to 2857223 tid 3 2023-06-27T12:41:00.591 INFO:tasks.rados.rados.0.smithi138.stdout:597: write (excl) oid 6 current snap is 78 2023-06-27T12:41:00.592 INFO:tasks.rados.rados.0.smithi138.stdout:597: seq_num 290 ranges {474030=643624,1520131=735769} 2023-06-27T12:41:00.592 INFO:tasks.rados.rados.0.smithi138.stdout:597: writing smithi138138488-6 from 474030 to 1117654 tid 1 2023-06-27T12:41:00.599 INFO:tasks.rados.rados.0.smithi138.stdout:597: writing smithi138138488-6 from 1520131 to 2255900 tid 2 2023-06-27T12:41:00.599 INFO:tasks.rados.rados.0.smithi138.stdout:594: expect (ObjNum 258 snap 70 seq_num 258) 2023-06-27T12:41:00.761 INFO:tasks.rados.rados.0.smithi138.stdout:595: finishing write tid 1 to smithi138138488-30 2023-06-27T12:41:00.762 INFO:tasks.rados.rados.0.smithi138.stdout:595: finishing write tid 2 to smithi138138488-30 2023-06-27T12:41:00.762 INFO:tasks.rados.rados.0.smithi138.stdout:595: finishing write tid 3 to smithi138138488-30 2023-06-27T12:41:00.762 INFO:tasks.rados.rados.0.smithi138.stdout:598: read oid 39 snap 76 2023-06-27T12:41:00.763 INFO:tasks.rados.rados.0.smithi138.stdout:598: expect (ObjNum 230 snap 55 seq_num 230) 2023-06-27T12:41:00.763 INFO:tasks.rados.rados.0.smithi138.stdout:594: done (7 left) 2023-06-27T12:41:00.763 INFO:tasks.rados.rados.0.smithi138.stdout:599: delete oid 12 current snap is 78 2023-06-27T12:41:00.764 INFO:tasks.rados.rados.0.smithi138.stdout:595: finishing write tid 4 to smithi138138488-30 2023-06-27T12:41:00.764 INFO:tasks.rados.rados.0.smithi138.stdout:595: oid 30 updating version 0 to 76 2023-06-27T12:41:00.764 INFO:tasks.rados.rados.0.smithi138.stdout:595: oid 30 version 76 is already newer than 76 2023-06-27T12:41:00.764 INFO:tasks.rados.rados.0.smithi138.stdout:595: oid 30 version 76 is already newer than 76 2023-06-27T12:41:00.765 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 76 (ObjNum 288 snap 78 seq_num 288) dirty exists 2023-06-27T12:41:00.765 INFO:tasks.rados.rados.0.smithi138.stdout:595: left oid 30 (ObjNum 288 snap 78 seq_num 288) 2023-06-27T12:41:00.765 INFO:tasks.rados.rados.0.smithi138.stdout:590: finishing write tid 1 to smithi138138488-2 2023-06-27T12:41:00.766 INFO:tasks.rados.rados.0.smithi138.stdout:597: finishing write tid 1 to smithi138138488-6 2023-06-27T12:41:00.766 INFO:tasks.rados.rados.0.smithi138.stdout:596: finishing write tid 1 to smithi138138488-5 2023-06-27T12:41:00.766 INFO:tasks.rados.rados.0.smithi138.stdout:590: finishing write tid 2 to smithi138138488-2 2023-06-27T12:41:00.766 INFO:tasks.rados.rados.0.smithi138.stdout:590: finishing write tid 3 to smithi138138488-2 2023-06-27T12:41:00.766 INFO:tasks.rados.rados.0.smithi138.stdout:590: finishing write tid 4 to smithi138138488-2 2023-06-27T12:41:00.767 INFO:tasks.rados.rados.0.smithi138.stdout:590: oid 2 updating version 0 to 236 2023-06-27T12:41:00.767 INFO:tasks.rados.rados.0.smithi138.stdout:590: oid 2 version 236 is already newer than 236 2023-06-27T12:41:00.767 INFO:tasks.rados.rados.0.smithi138.stdout:590: oid 2 version 236 is already newer than 234 2023-06-27T12:41:00.767 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 236 (ObjNum 285 snap 78 seq_num 285) dirty exists 2023-06-27T12:41:00.768 INFO:tasks.rados.rados.0.smithi138.stdout:590: left oid 2 (ObjNum 285 snap 78 seq_num 285) 2023-06-27T12:41:00.768 INFO:tasks.rados.rados.0.smithi138.stdout:591: expect (ObjNum 271 snap 70 seq_num 271) 2023-06-27T12:41:00.863 INFO:tasks.rados.rados.0.smithi138.stdout:597: finishing write tid 2 to smithi138138488-6 2023-06-27T12:41:00.863 INFO:tasks.rados.rados.0.smithi138.stdout:597: finishing write tid 3 to smithi138138488-6 2023-06-27T12:41:00.863 INFO:tasks.rados.rados.0.smithi138.stdout:597: finishing write tid 4 to smithi138138488-6 2023-06-27T12:41:00.863 INFO:tasks.rados.rados.0.smithi138.stdout:597: oid 6 updating version 0 to 80 2023-06-27T12:41:00.863 INFO:tasks.rados.rados.0.smithi138.stdout:597: oid 6 version 80 is already newer than 77 2023-06-27T12:41:00.863 INFO:tasks.rados.rados.0.smithi138.stdout:597: oid 6 version 80 is already newer than 80 2023-06-27T12:41:00.864 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 80 (ObjNum 290 snap 78 seq_num 290) dirty exists 2023-06-27T12:41:00.864 INFO:tasks.rados.rados.0.smithi138.stdout:597: left oid 6 (ObjNum 290 snap 78 seq_num 290) 2023-06-27T12:41:00.864 INFO:tasks.rados.rados.0.smithi138.stdout:593: finishing write tid 1 to smithi138138488-32 2023-06-27T12:41:00.865 INFO:tasks.rados.rados.0.smithi138.stdout:596: finishing write tid 2 to smithi138138488-5 2023-06-27T12:41:00.865 INFO:tasks.rados.rados.0.smithi138.stdout:596: finishing write tid 3 to smithi138138488-5 2023-06-27T12:41:00.865 INFO:tasks.rados.rados.0.smithi138.stdout:596: finishing write tid 4 to smithi138138488-5 2023-06-27T12:41:00.865 INFO:tasks.rados.rados.0.smithi138.stdout:590: done (7 left) 2023-06-27T12:41:00.865 INFO:tasks.rados.rados.0.smithi138.stdout:591: done (6 left) 2023-06-27T12:41:00.865 INFO:tasks.rados.rados.0.smithi138.stdout:595: done (5 left) 2023-06-27T12:41:00.866 INFO:tasks.rados.rados.0.smithi138.stdout:597: done (4 left) 2023-06-27T12:41:00.866 INFO:tasks.rados.rados.0.smithi138.stdout:599: done (3 left) 2023-06-27T12:41:00.866 INFO:tasks.rados.rados.0.smithi138.stdout:600: snap_remove snap 71 2023-06-27T12:41:01.463 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:01.471 INFO:tasks.rados.rados.0.smithi138.stdout:600: done (3 left) 2023-06-27T12:41:01.471 INFO:tasks.rados.rados.0.smithi138.stdout:601: snap_create 2023-06-27T12:41:01.471 INFO:tasks.rados.rados.0.smithi138.stdout:596: finishing write tid 5 to smithi138138488-5 2023-06-27T12:41:01.471 INFO:tasks.rados.rados.0.smithi138.stdout:596: oid 5 updating version 0 to 81 2023-06-27T12:41:01.471 INFO:tasks.rados.rados.0.smithi138.stdout:596: oid 5 version 81 is already newer than 81 2023-06-27T12:41:01.472 INFO:tasks.rados.rados.0.smithi138.stdout:596: oid 5 version 81 is already newer than 77 2023-06-27T12:41:01.472 INFO:tasks.rados.rados.0.smithi138.stdout:596: oid 5 version 81 is already newer than 81 2023-06-27T12:41:01.472 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 5 v 81 (ObjNum 289 snap 78 seq_num 289) dirty exists 2023-06-27T12:41:01.472 INFO:tasks.rados.rados.0.smithi138.stdout:596: left oid 5 (ObjNum 289 snap 78 seq_num 289) 2023-06-27T12:41:01.472 INFO:tasks.rados.rados.0.smithi138.stdout:598: expect (ObjNum 230 snap 55 seq_num 230) 2023-06-27T12:41:01.593 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:01.588+0000 7fdc08fea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:01.596 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:01.592+0000 7fdc08fea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:01.625 INFO:tasks.rados.rados.0.smithi138.stdout:593: finishing write tid 2 to smithi138138488-32 2023-06-27T12:41:01.625 INFO:tasks.rados.rados.0.smithi138.stdout:593: finishing write tid 3 to smithi138138488-32 2023-06-27T12:41:01.625 INFO:tasks.rados.rados.0.smithi138.stdout:593: finishing write tid 4 to smithi138138488-32 2023-06-27T12:41:01.625 INFO:tasks.rados.rados.0.smithi138.stdout:593: finishing write tid 5 to smithi138138488-32 2023-06-27T12:41:01.625 INFO:tasks.rados.rados.0.smithi138.stdout:593: oid 32 updating version 0 to 241 2023-06-27T12:41:01.625 INFO:tasks.rados.rados.0.smithi138.stdout:593: oid 32 version 241 is already newer than 241 2023-06-27T12:41:01.626 INFO:tasks.rados.rados.0.smithi138.stdout:593: oid 32 version 241 is already newer than 241 2023-06-27T12:41:01.626 INFO:tasks.rados.rados.0.smithi138.stdout:593: oid 32 version 241 is already newer than 237 2023-06-27T12:41:01.626 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 241 (ObjNum 287 snap 78 seq_num 287) dirty exists 2023-06-27T12:41:01.626 INFO:tasks.rados.rados.0.smithi138.stdout:593: left oid 32 (ObjNum 287 snap 78 seq_num 287) 2023-06-27T12:41:01.640 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:01.643 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:01.643 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:02.478 INFO:tasks.rados.rados.0.smithi138.stdout:593: done (3 left) 2023-06-27T12:41:02.478 INFO:tasks.rados.rados.0.smithi138.stdout:596: done (2 left) 2023-06-27T12:41:02.478 INFO:tasks.rados.rados.0.smithi138.stdout:598: done (1 left) 2023-06-27T12:41:02.478 INFO:tasks.rados.rados.0.smithi138.stdout:601: done (0 left) 2023-06-27T12:41:02.478 INFO:tasks.rados.rados.0.smithi138.stdout:602: snap_create 2023-06-27T12:41:02.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~28s 2023-06-27T12:41:03.479 INFO:tasks.rados.rados.0.smithi138.stdout:602: done (0 left) 2023-06-27T12:41:03.480 INFO:tasks.rados.rados.0.smithi138.stdout:603: write oid 29 current snap is 80 2023-06-27T12:41:03.480 INFO:tasks.rados.rados.0.smithi138.stdout:603: seq_num 292 ranges {507644=794122,1920742=623085} 2023-06-27T12:41:03.487 INFO:tasks.rados.rados.0.smithi138.stdout:603: writing smithi138138488-29 from 507644 to 1301766 tid 1 2023-06-27T12:41:03.494 INFO:tasks.rados.rados.0.smithi138.stdout:603: writing smithi138138488-29 from 1920742 to 2543827 tid 2 2023-06-27T12:41:03.494 INFO:tasks.rados.rados.0.smithi138.stdout:604: write (excl) oid 19 current snap is 80 2023-06-27T12:41:03.494 INFO:tasks.rados.rados.0.smithi138.stdout:604: seq_num 293 ranges {524451=469371,1579224=483096,2529878=394573} 2023-06-27T12:41:03.499 INFO:tasks.rados.rados.0.smithi138.stdout:604: writing smithi138138488-19 from 524451 to 993822 tid 1 2023-06-27T12:41:03.505 INFO:tasks.rados.rados.0.smithi138.stdout:604: writing smithi138138488-19 from 1579224 to 2062320 tid 2 2023-06-27T12:41:03.512 INFO:tasks.rados.rados.0.smithi138.stdout:604: writing smithi138138488-19 from 2529878 to 2924451 tid 3 2023-06-27T12:41:03.512 INFO:tasks.rados.rados.0.smithi138.stdout:605: write (excl) oid 31 current snap is 80 2023-06-27T12:41:03.512 INFO:tasks.rados.rados.0.smithi138.stdout:605: seq_num 294 ranges {541258=544620,1637706=626754,2840948=100310} 2023-06-27T12:41:03.519 INFO:tasks.rados.rados.0.smithi138.stdout:605: writing smithi138138488-31 from 541258 to 1085878 tid 1 2023-06-27T12:41:03.525 INFO:tasks.rados.rados.0.smithi138.stdout:605: writing smithi138138488-31 from 1637706 to 2264460 tid 2 2023-06-27T12:41:03.530 INFO:tasks.rados.rados.0.smithi138.stdout:605: writing smithi138138488-31 from 2840948 to 2941258 tid 3 2023-06-27T12:41:03.531 INFO:tasks.rados.rados.0.smithi138.stdout:606: read oid 22 snap -1 2023-06-27T12:41:03.531 INFO:tasks.rados.rados.0.smithi138.stdout:606: expect (ObjNum 275 snap 72 seq_num 275) 2023-06-27T12:41:03.531 INFO:tasks.rados.rados.0.smithi138.stdout:607: read oid 12 snap 51 2023-06-27T12:41:03.531 INFO:tasks.rados.rados.0.smithi138.stdout:607: expect deleted 2023-06-27T12:41:03.531 INFO:tasks.rados.rados.0.smithi138.stdout:608: read oid 9 snap -1 2023-06-27T12:41:03.531 INFO:tasks.rados.rados.0.smithi138.stdout:608: expect (ObjNum 270 snap 70 seq_num 270) 2023-06-27T12:41:03.532 INFO:tasks.rados.rados.0.smithi138.stdout:609: write (excl) oid 24 current snap is 80 2023-06-27T12:41:03.532 INFO:tasks.rados.rados.0.smithi138.stdout:609: seq_num 295 ranges {558065=619869,1579835=486765,2468371=489694} 2023-06-27T12:41:03.537 INFO:tasks.rados.rados.0.smithi138.stdout:609: writing smithi138138488-24 from 558065 to 1177934 tid 1 2023-06-27T12:41:03.543 INFO:tasks.rados.rados.0.smithi138.stdout:609: writing smithi138138488-24 from 1579835 to 2066600 tid 2 2023-06-27T12:41:03.550 INFO:tasks.rados.rados.0.smithi138.stdout:609: writing smithi138138488-24 from 2468371 to 2958065 tid 3 2023-06-27T12:41:03.550 INFO:tasks.rados.rados.0.smithi138.stdout:610: read oid 41 snap -1 2023-06-27T12:41:03.550 INFO:tasks.rados.rados.0.smithi138.stdout:610: expect (ObjNum 268 snap 70 seq_num 268) 2023-06-27T12:41:03.550 INFO:tasks.rados.rados.0.smithi138.stdout:611: read oid 17 snap -1 2023-06-27T12:41:03.550 INFO:tasks.rados.rados.0.smithi138.stdout:611: expect (ObjNum 219 snap 49 seq_num 219) 2023-06-27T12:41:03.551 INFO:tasks.rados.rados.0.smithi138.stdout:612: delete oid 40 current snap is 80 2023-06-27T12:41:03.641 INFO:tasks.rados.rados.0.smithi138.stdout:606: expect (ObjNum 275 snap 72 seq_num 275) 2023-06-27T12:41:03.644 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:03.791 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:03.786+0000 7fd7a4fb4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:03.791+0000 7fd7a4fb4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:03.808 INFO:tasks.rados.rados.0.smithi138.stdout:606: done (9 left) 2023-06-27T12:41:03.809 INFO:tasks.rados.rados.0.smithi138.stdout:612: done (8 left) 2023-06-27T12:41:03.809 INFO:tasks.rados.rados.0.smithi138.stdout:613: read oid 47 snap 67 2023-06-27T12:41:03.809 INFO:tasks.rados.rados.0.smithi138.stdout:613: expect (ObjNum 192 snap 38 seq_num 192) 2023-06-27T12:41:03.809 INFO:tasks.rados.rados.0.smithi138.stdout:614: write (excl) oid 28 current snap is 80 2023-06-27T12:41:03.809 INFO:tasks.rados.rados.0.smithi138.stdout:605: finishing write tid 1 to smithi138138488-31 2023-06-27T12:41:03.809 INFO:tasks.rados.rados.0.smithi138.stdout:603: finishing write tid 1 to smithi138138488-29 2023-06-27T12:41:03.810 INFO:tasks.rados.rados.0.smithi138.stdout:603: finishing write tid 2 to smithi138138488-29 2023-06-27T12:41:03.810 INFO:tasks.rados.rados.0.smithi138.stdout:603: finishing write tid 3 to smithi138138488-29 2023-06-27T12:41:03.810 INFO:tasks.rados.rados.0.smithi138.stdout:603: finishing write tid 4 to smithi138138488-29 2023-06-27T12:41:03.810 INFO:tasks.rados.rados.0.smithi138.stdout:603: oid 29 updating version 0 to 53 2023-06-27T12:41:03.810 INFO:tasks.rados.rados.0.smithi138.stdout:603: oid 29 version 53 is already newer than 53 2023-06-27T12:41:03.810 INFO:tasks.rados.rados.0.smithi138.stdout:603: oid 29 version 53 is already newer than 53 2023-06-27T12:41:03.810 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 53 (ObjNum 292 snap 80 seq_num 292) dirty exists 2023-06-27T12:41:03.810 INFO:tasks.rados.rados.0.smithi138.stdout:603: left oid 29 (ObjNum 292 snap 80 seq_num 292) 2023-06-27T12:41:03.811 INFO:tasks.rados.rados.0.smithi138.stdout:611: expect (ObjNum 219 snap 49 seq_num 219) 2023-06-27T12:41:03.880 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:03.883 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:03.883 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:03.909 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:03.909 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:03.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:04.039 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:04.032+0000 7f7a96396700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:04.038+0000 7f7a96396700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:04.098 INFO:tasks.rados.rados.0.smithi138.stdout:604: finishing write tid 1 to smithi138138488-19 2023-06-27T12:41:04.099 INFO:tasks.rados.rados.0.smithi138.stdout:614: seq_num 297 ranges {591679=486720,1813152=490434} 2023-06-27T12:41:04.104 INFO:tasks.rados.rados.0.smithi138.stdout:614: writing smithi138138488-28 from 591679 to 1078399 tid 1 2023-06-27T12:41:04.111 INFO:tasks.rados.rados.0.smithi138.stdout:614: writing smithi138138488-28 from 1813152 to 2303586 tid 2 2023-06-27T12:41:04.111 INFO:tasks.rados.rados.0.smithi138.stdout:603: done (9 left) 2023-06-27T12:41:04.112 INFO:tasks.rados.rados.0.smithi138.stdout:607: done (8 left) 2023-06-27T12:41:04.112 INFO:tasks.rados.rados.0.smithi138.stdout:611: done (7 left) 2023-06-27T12:41:04.112 INFO:tasks.rados.rados.0.smithi138.stdout:615: delete oid 7 current snap is 80 2023-06-27T12:41:04.112 INFO:tasks.rados.rados.0.smithi138.stdout:610: expect (ObjNum 268 snap 70 seq_num 268) 2023-06-27T12:41:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:04.245 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:04.256 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2023-06-27T12:41:03.942559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105181055,"num_objects":198,"num_object_clones":140,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":979,"num_read_kb":624465,"num_write":1102,"num_write_kb":280937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":691788,"kb_used_data":568108,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376795572,"statfs":{"total":386547056640,"available":385838665728,"internally_reserved":0,"allocated":581742592,"data_stored":676852329,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000844"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":323,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583467+0000","last_change":"2023-06-27T12:40:33.819159+0000","last_active":"2023-06-27T12:40:36.583467+0000","last_peered":"2023-06-27T12:40:36.583467+0000","last_clean":"2023-06-27T12:40:36.583467+0000","last_became_active":"2023-06-27T12:38:04.537739+0000","last_became_peered":"2023-06-27T12:38:04.537739+0000","last_unstale":"2023-06-27T12:40:36.583467+0000","last_undegraded":"2023-06-27T12:40:36.583467+0000","last_fullsized":"2023-06-27T12:40:36.583467+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":439,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.442065+0000","last_change":"2023-06-27T12:40:33.439175+0000","last_active":"2023-06-27T12:40:36.442065+0000","last_peered":"2023-06-27T12:40:36.442065+0000","last_clean":"2023-06-27T12:40:36.442065+0000","last_became_active":"2023-06-27T12:38:04.575466+0000","last_became_peered":"2023-06-27T12:38:04.575466+0000","last_unstale":"2023-06-27T12:40:36.442065+0000","last_undegraded":"2023-06-27T12:40:36.442065+0000","last_fullsized":"2023-06-27T12:40:36.442065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":62,"num_read_kb":68582,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":444,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.577791+0000","last_change":"2023-06-27T12:40:33.461948+0000","last_active":"2023-06-27T12:40:36.577791+0000","last_peered":"2023-06-27T12:40:36.577791+0000","last_clean":"2023-06-27T12:40:36.577791+0000","last_became_active":"2023-06-27T12:38:04.526467+0000","last_became_peered":"2023-06-27T12:38:04.526467+0000","last_unstale":"2023-06-27T12:40:36.577791+0000","last_undegraded":"2023-06-27T12:40:36.577791+0000","last_fullsized":"2023-06-27T12:40:36.577791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":20453,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'67","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591295+0000","last_change":"2023-06-27T12:40:33.824008+0000","last_active":"2023-06-27T12:40:36.591295+0000","last_peered":"2023-06-27T12:40:36.591295+0000","last_clean":"2023-06-27T12:40:36.591295+0000","last_became_active":"2023-06-27T12:38:04.519539+0000","last_became_peered":"2023-06-27T12:38:04.519539+0000","last_unstale":"2023-06-27T12:40:36.591295+0000","last_undegraded":"2023-06-27T12:40:36.591295+0000","last_fullsized":"2023-06-27T12:40:36.591295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050553,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":27199,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":348,"reported_epoch":160,"state":"stale+active+clean","last_fresh":"2023-06-27T12:40:28.454098+0000","last_change":"2023-06-27T12:40:27.808791+0000","last_active":"2023-06-27T12:40:28.454098+0000","last_peered":"2023-06-27T12:40:28.454098+0000","last_clean":"2023-06-27T12:40:28.454098+0000","last_became_active":"2023-06-27T12:38:04.567268+0000","last_became_peered":"2023-06-27T12:38:04.567268+0000","last_unstale":"2023-06-27T12:40:59.452040+0000","last_undegraded":"2023-06-27T12:40:28.454098+0000","last_fullsized":"2023-06-27T12:40:28.454098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294172,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":25,"num_read_kb":28579,"num_write":26,"num_write_kb":7078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"147'167","reported_seq":497,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.583253+0000","last_change":"2023-06-27T12:40:33.818274+0000","last_active":"2023-06-27T12:40:36.583253+0000","last_peered":"2023-06-27T12:40:36.583253+0000","last_clean":"2023-06-27T12:40:36.583253+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:40:36.583253+0000","last_undegraded":"2023-06-27T12:40:36.583253+0000","last_fullsized":"2023-06-27T12:40:36.583253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538088,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":113812,"num_write":108,"num_write_kb":31100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"163'105","reported_seq":406,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.594191+0000","last_change":"2023-06-27T12:40:33.883382+0000","last_active":"2023-06-27T12:40:36.594191+0000","last_peered":"2023-06-27T12:40:36.594191+0000","last_clean":"2023-06-27T12:40:36.594191+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:40:36.594191+0000","last_undegraded":"2023-06-27T12:40:36.594191+0000","last_fullsized":"2023-06-27T12:40:36.594191+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227043,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":57,"num_read_kb":30118,"num_write":63,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":370,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.581982+0000","last_change":"2023-06-27T12:40:33.451340+0000","last_active":"2023-06-27T12:40:36.581982+0000","last_peered":"2023-06-27T12:40:36.581982+0000","last_clean":"2023-06-27T12:40:36.581982+0000","last_became_active":"2023-06-27T12:38:04.579741+0000","last_became_peered":"2023-06-27T12:38:04.579741+0000","last_unstale":"2023-06-27T12:40:36.581982+0000","last_undegraded":"2023-06-27T12:40:36.581982+0000","last_fullsized":"2023-06-27T12:40:36.581982+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2773188,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.438400+0000","last_change":"2023-06-27T12:37:57.907758+0000","last_active":"2023-06-27T12:40:36.438400+0000","last_peered":"2023-06-27T12:40:36.438400+0000","last_clean":"2023-06-27T12:40:36.438400+0000","last_became_active":"2023-06-27T12:37:57.904736+0000","last_became_peered":"2023-06-27T12:37:57.904736+0000","last_unstale":"2023-06-27T12:40:36.438400+0000","last_undegraded":"2023-06-27T12:40:36.438400+0000","last_fullsized":"2023-06-27T12:40:36.438400+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":318,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.582504+0000","last_change":"2023-06-27T12:40:33.461555+0000","last_active":"2023-06-27T12:40:36.582504+0000","last_peered":"2023-06-27T12:40:36.582504+0000","last_clean":"2023-06-27T12:40:36.582504+0000","last_became_active":"2023-06-27T12:38:04.511886+0000","last_became_peered":"2023-06-27T12:38:04.511886+0000","last_unstale":"2023-06-27T12:40:36.582504+0000","last_undegraded":"2023-06-27T12:40:36.582504+0000","last_fullsized":"2023-06-27T12:40:36.582504+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591210+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:40:36.591210+0000","last_peered":"2023-06-27T12:40:36.591210+0000","last_clean":"2023-06-27T12:40:36.591210+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:40:36.591210+0000","last_undegraded":"2023-06-27T12:40:36.591210+0000","last_fullsized":"2023-06-27T12:40:36.591210+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"158'76","reported_seq":395,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.443043+0000","last_change":"2023-06-27T12:40:33.438763+0000","last_active":"2023-06-27T12:40:36.443043+0000","last_peered":"2023-06-27T12:40:36.443043+0000","last_clean":"2023-06-27T12:40:36.443043+0000","last_became_active":"2023-06-27T12:38:04.549531+0000","last_became_peered":"2023-06-27T12:38:04.549531+0000","last_unstale":"2023-06-27T12:40:36.443043+0000","last_undegraded":"2023-06-27T12:40:36.443043+0000","last_fullsized":"2023-06-27T12:40:36.443043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689285,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":36,"num_read_kb":31324,"num_write":50,"num_write_kb":13765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.587199+0000","last_change":"2023-06-27T12:37:57.882217+0000","last_active":"2023-06-27T12:40:36.587199+0000","last_peered":"2023-06-27T12:40:36.587199+0000","last_clean":"2023-06-27T12:40:36.587199+0000","last_became_active":"2023-06-27T12:37:57.878723+0000","last_became_peered":"2023-06-27T12:37:57.878723+0000","last_unstale":"2023-06-27T12:40:36.587199+0000","last_undegraded":"2023-06-27T12:40:36.587199+0000","last_fullsized":"2023-06-27T12:40:36.587199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"160'233","reported_seq":486,"reported_epoch":160,"state":"stale+active+clean","last_fresh":"2023-06-27T12:40:28.533100+0000","last_change":"2023-06-27T12:40:28.538999+0000","last_active":"2023-06-27T12:40:28.533100+0000","last_peered":"2023-06-27T12:40:28.533100+0000","last_clean":"2023-06-27T12:40:28.533100+0000","last_became_active":"2023-06-27T12:38:04.591604+0000","last_became_peered":"2023-06-27T12:38:04.591604+0000","last_unstale":"2023-06-27T12:40:59.452042+0000","last_undegraded":"2023-06-27T12:40:28.533100+0000","last_fullsized":"2023-06-27T12:40:28.533100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10319597,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":71887,"num_write":132,"num_write_kb":39413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.575764+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:40:36.575764+0000","last_peered":"2023-06-27T12:40:36.575764+0000","last_clean":"2023-06-27T12:40:36.575764+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:40:36.575764+0000","last_undegraded":"2023-06-27T12:40:36.575764+0000","last_fullsized":"2023-06-27T12:40:36.575764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":372,"reported_epoch":160,"state":"stale+active+clean","last_fresh":"2023-06-27T12:40:28.449489+0000","last_change":"2023-06-27T12:37:55.863331+0000","last_active":"2023-06-27T12:40:28.449489+0000","last_peered":"2023-06-27T12:40:28.449489+0000","last_clean":"2023-06-27T12:40:28.449489+0000","last_became_active":"2023-06-27T12:37:55.859145+0000","last_became_peered":"2023-06-27T12:37:55.859145+0000","last_unstale":"2023-06-27T12:40:59.452044+0000","last_undegraded":"2023-06-27T12:40:28.449489+0000","last_fullsized":"2023-06-27T12:40:28.449489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"164'175","reported_seq":504,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.578849+0000","last_change":"2023-06-27T12:40:33.450942+0000","last_active":"2023-06-27T12:40:36.578849+0000","last_peered":"2023-06-27T12:40:36.578849+0000","last_clean":"2023-06-27T12:40:36.578849+0000","last_became_active":"2023-06-27T12:38:04.531634+0000","last_became_peered":"2023-06-27T12:38:04.531634+0000","last_unstale":"2023-06-27T12:40:36.578849+0000","last_undegraded":"2023-06-27T12:40:36.578849+0000","last_fullsized":"2023-06-27T12:40:36.578849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15025524,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":70272,"num_write":126,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.585760+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:40:36.585760+0000","last_peered":"2023-06-27T12:40:36.585760+0000","last_clean":"2023-06-27T12:40:36.585760+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:40:36.585760+0000","last_undegraded":"2023-06-27T12:40:36.585760+0000","last_fullsized":"2023-06-27T12:40:36.585760+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"151'73","reported_seq":376,"reported_epoch":160,"state":"stale+active+clean","last_fresh":"2023-06-27T12:40:28.446866+0000","last_change":"2023-06-27T12:40:27.809749+0000","last_active":"2023-06-27T12:40:28.446866+0000","last_peered":"2023-06-27T12:40:28.446866+0000","last_clean":"2023-06-27T12:40:28.446866+0000","last_became_active":"2023-06-27T12:38:04.573615+0000","last_became_peered":"2023-06-27T12:38:04.573615+0000","last_unstale":"2023-06-27T12:40:59.452046+0000","last_undegraded":"2023-06-27T12:40:28.446866+0000","last_fullsized":"2023-06-27T12:40:28.446866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840548,"num_objects":11,"num_object_clones":5,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":33,"num_read_kb":24675,"num_write":53,"num_write_kb":15374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":160,"state":"stale+active+clean","last_fresh":"2023-06-27T12:40:28.447682+0000","last_change":"2023-06-27T12:37:57.934796+0000","last_active":"2023-06-27T12:40:28.447682+0000","last_peered":"2023-06-27T12:40:28.447682+0000","last_clean":"2023-06-27T12:40:28.447682+0000","last_became_active":"2023-06-27T12:37:57.931849+0000","last_became_peered":"2023-06-27T12:37:57.931849+0000","last_unstale":"2023-06-27T12:40:59.452048+0000","last_undegraded":"2023-06-27T12:40:28.447682+0000","last_fullsized":"2023-06-27T12:40:28.447682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"147'86","reported_seq":417,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.448935+0000","last_change":"2023-06-27T12:40:33.436804+0000","last_active":"2023-06-27T12:40:36.448935+0000","last_peered":"2023-06-27T12:40:36.448935+0000","last_clean":"2023-06-27T12:40:36.448935+0000","last_became_active":"2023-06-27T12:38:04.586364+0000","last_became_peered":"2023-06-27T12:38:04.586364+0000","last_unstale":"2023-06-27T12:40:36.448935+0000","last_undegraded":"2023-06-27T12:40:36.448935+0000","last_fullsized":"2023-06-27T12:40:36.448935+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101073,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":42853,"num_write":68,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"117'49","reported_seq":355,"reported_epoch":160,"state":"stale+active+clean","last_fresh":"2023-06-27T12:40:28.455948+0000","last_change":"2023-06-27T12:40:27.810261+0000","last_active":"2023-06-27T12:40:28.455948+0000","last_peered":"2023-06-27T12:40:28.455948+0000","last_clean":"2023-06-27T12:40:28.455948+0000","last_became_active":"2023-06-27T12:38:04.557037+0000","last_became_peered":"2023-06-27T12:38:04.557037+0000","last_unstale":"2023-06-27T12:40:59.452049+0000","last_undegraded":"2023-06-27T12:40:28.455948+0000","last_fullsized":"2023-06-27T12:40:28.455948+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6100941,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":17648,"num_write":35,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":259,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.449460+0000","last_change":"2023-06-27T12:37:57.914235+0000","last_active":"2023-06-27T12:40:36.449460+0000","last_peered":"2023-06-27T12:40:36.449460+0000","last_clean":"2023-06-27T12:40:36.449460+0000","last_became_active":"2023-06-27T12:37:57.911304+0000","last_became_peered":"2023-06-27T12:37:57.911304+0000","last_unstale":"2023-06-27T12:40:36.449460+0000","last_undegraded":"2023-06-27T12:40:36.449460+0000","last_fullsized":"2023-06-27T12:40:36.449460+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":253,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.591666+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:40:36.591666+0000","last_peered":"2023-06-27T12:40:36.591666+0000","last_clean":"2023-06-27T12:40:36.591666+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:40:36.591666+0000","last_undegraded":"2023-06-27T12:40:36.591666+0000","last_fullsized":"2023-06-27T12:40:36.591666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":415,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-06-27T12:40:36.436802+0000","last_change":"2023-06-27T12:40:33.436080+0000","last_active":"2023-06-27T12:40:36.436802+0000","last_peered":"2023-06-27T12:40:36.436802+0000","last_clean":"2023-06-27T12:40:36.436802+0000","last_became_active":"2023-06-27T12:38:04.532943+0000","last_became_peered":"2023-06-27T12:38:04.532943+0000","last_unstale":"2023-06-27T12:40:36.436802+0000","last_undegraded":"2023-06-27T12:40:36.436802+0000","last_fullsized":"2023-06-27T12:40:36.436802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":42938,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":104590687,"num_objects":196,"num_object_clones":140,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":876,"num_read_kb":624374,"num_write":965,"num_write_kb":279043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":160452,"kb_used_data":129532,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94211388,"statfs":{"total":96636764160,"available":96472461312,"internally_reserved":0,"allocated":132640768,"data_stored":154644533,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167772,"kb_used_data":136852,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204068,"statfs":{"total":96636764160,"available":96464965632,"internally_reserved":0,"allocated":140136448,"data_stored":162212424,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":182284,"kb_used_data":151364,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94189556,"statfs":{"total":96636764160,"available":96450105344,"internally_reserved":0,"allocated":154996736,"data_stored":182955820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:04.287 INFO:tasks.rados.rados.0.smithi138.stdout:605: finishing write tid 2 to smithi138138488-31 2023-06-27T12:41:04.288 INFO:tasks.rados.rados.0.smithi138.stdout:605: finishing write tid 3 to smithi138138488-31 2023-06-27T12:41:04.288 INFO:tasks.rados.rados.0.smithi138.stdout:605: finishing write tid 4 to smithi138138488-31 2023-06-27T12:41:04.288 INFO:tasks.rados.rados.0.smithi138.stdout:605: finishing write tid 5 to smithi138138488-31 2023-06-27T12:41:04.288 INFO:tasks.rados.rados.0.smithi138.stdout:605: oid 31 updating version 0 to 192 2023-06-27T12:41:04.288 INFO:tasks.rados.rados.0.smithi138.stdout:605: oid 31 updating version 192 to 196 2023-06-27T12:41:04.288 INFO:tasks.rados.rados.0.smithi138.stdout:605: oid 31 version 196 is already newer than 196 2023-06-27T12:41:04.288 INFO:tasks.rados.rados.0.smithi138.stdout:605: oid 31 version 196 is already newer than 196 2023-06-27T12:41:04.289 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 196 (ObjNum 294 snap 80 seq_num 294) dirty exists 2023-06-27T12:41:04.289 INFO:tasks.rados.rados.0.smithi138.stdout:605: left oid 31 (ObjNum 294 snap 80 seq_num 294) 2023-06-27T12:41:04.289 INFO:tasks.rados.rados.0.smithi138.stdout:604: finishing write tid 2 to smithi138138488-19 2023-06-27T12:41:04.289 INFO:tasks.rados.rados.0.smithi138.stdout:604: finishing write tid 3 to smithi138138488-19 2023-06-27T12:41:04.289 INFO:tasks.rados.rados.0.smithi138.stdout:609: finishing write tid 1 to smithi138138488-24 2023-06-27T12:41:04.289 INFO:tasks.rados.rados.0.smithi138.stdout:604: finishing write tid 4 to smithi138138488-19 2023-06-27T12:41:04.289 INFO:tasks.rados.rados.0.smithi138.stdout:604: finishing write tid 5 to smithi138138488-19 2023-06-27T12:41:04.290 INFO:tasks.rados.rados.0.smithi138.stdout:604: oid 19 updating version 0 to 180 2023-06-27T12:41:04.290 INFO:tasks.rados.rados.0.smithi138.stdout:604: oid 19 version 180 is already newer than 180 2023-06-27T12:41:04.290 INFO:tasks.rados.rados.0.smithi138.stdout:604: oid 19 version 180 is already newer than 180 2023-06-27T12:41:04.290 INFO:tasks.rados.rados.0.smithi138.stdout:604: oid 19 version 180 is already newer than 176 2023-06-27T12:41:04.290 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 180 (ObjNum 293 snap 80 seq_num 293) dirty exists 2023-06-27T12:41:04.290 INFO:tasks.rados.rados.0.smithi138.stdout:604: left oid 19 (ObjNum 293 snap 80 seq_num 293) 2023-06-27T12:41:04.290 INFO:tasks.rados.rados.0.smithi138.stdout:608: expect (ObjNum 270 snap 70 seq_num 270) 2023-06-27T12:41:04.499 INFO:tasks.rados.rados.0.smithi138.stdout:613: expect (ObjNum 192 snap 38 seq_num 192) 2023-06-27T12:41:04.655 INFO:tasks.rados.rados.0.smithi138.stdout:609: finishing write tid 2 to smithi138138488-24 2023-06-27T12:41:04.655 INFO:tasks.rados.rados.0.smithi138.stdout:609: finishing write tid 3 to smithi138138488-24 2023-06-27T12:41:04.655 INFO:tasks.rados.rados.0.smithi138.stdout:604: done (7 left) 2023-06-27T12:41:04.656 INFO:tasks.rados.rados.0.smithi138.stdout:605: done (6 left) 2023-06-27T12:41:04.656 INFO:tasks.rados.rados.0.smithi138.stdout:608: done (5 left) 2023-06-27T12:41:04.656 INFO:tasks.rados.rados.0.smithi138.stdout:610: done (4 left) 2023-06-27T12:41:04.656 INFO:tasks.rados.rados.0.smithi138.stdout:613: done (3 left) 2023-06-27T12:41:04.656 INFO:tasks.rados.rados.0.smithi138.stdout:615: done (2 left) 2023-06-27T12:41:04.656 INFO:tasks.rados.rados.0.smithi138.stdout:616: write oid 46 current snap is 80 2023-06-27T12:41:04.656 INFO:tasks.rados.rados.0.smithi138.stdout:609: finishing write tid 4 to smithi138138488-24 2023-06-27T12:41:04.657 INFO:tasks.rados.rados.0.smithi138.stdout:609: finishing write tid 5 to smithi138138488-24 2023-06-27T12:41:04.657 INFO:tasks.rados.rados.0.smithi138.stdout:609: oid 24 updating version 0 to 197 2023-06-27T12:41:04.657 INFO:tasks.rados.rados.0.smithi138.stdout:609: oid 24 updating version 197 to 201 2023-06-27T12:41:04.657 INFO:tasks.rados.rados.0.smithi138.stdout:609: oid 24 version 201 is already newer than 201 2023-06-27T12:41:04.657 INFO:tasks.rados.rados.0.smithi138.stdout:609: oid 24 version 201 is already newer than 201 2023-06-27T12:41:04.658 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 201 (ObjNum 295 snap 80 seq_num 295) dirty exists 2023-06-27T12:41:04.658 INFO:tasks.rados.rados.0.smithi138.stdout:609: left oid 24 (ObjNum 295 snap 80 seq_num 295) 2023-06-27T12:41:04.658 INFO:tasks.rados.rados.0.smithi138.stdout:616: seq_num 299 ranges {625293=637218,1813763=494103,2978063=47230} 2023-06-27T12:41:04.661 INFO:tasks.rados.rados.0.smithi138.stdout:616: writing smithi138138488-46 from 625293 to 1262511 tid 1 2023-06-27T12:41:04.667 INFO:tasks.rados.rados.0.smithi138.stdout:616: writing smithi138138488-46 from 1813763 to 2307866 tid 2 2023-06-27T12:41:04.672 INFO:tasks.rados.rados.0.smithi138.stdout:616: writing smithi138138488-46 from 2978063 to 3025293 tid 3 2023-06-27T12:41:04.672 INFO:tasks.rados.rados.0.smithi138.stdout:609: done (2 left) 2023-06-27T12:41:04.672 INFO:tasks.rados.rados.0.smithi138.stdout:617: write oid 26 current snap is 80 2023-06-27T12:41:04.672 INFO:tasks.rados.rados.0.smithi138.stdout:614: finishing write tid 1 to smithi138138488-28 2023-06-27T12:41:04.673 INFO:tasks.rados.rados.0.smithi138.stdout:617: seq_num 300 ranges {642100=712467,1872245=637761} 2023-06-27T12:41:04.679 INFO:tasks.rados.rados.0.smithi138.stdout:617: writing smithi138138488-26 from 642100 to 1354567 tid 1 2023-06-27T12:41:04.686 INFO:tasks.rados.rados.0.smithi138.stdout:617: writing smithi138138488-26 from 1872245 to 2510006 tid 2 2023-06-27T12:41:04.686 INFO:tasks.rados.rados.0.smithi138.stdout:618: delete oid 31 current snap is 80 2023-06-27T12:41:04.686 INFO:tasks.rados.rados.0.smithi138.stdout:614: finishing write tid 2 to smithi138138488-28 2023-06-27T12:41:04.686 INFO:tasks.rados.rados.0.smithi138.stdout:614: finishing write tid 3 to smithi138138488-28 2023-06-27T12:41:04.686 INFO:tasks.rados.rados.0.smithi138.stdout:614: finishing write tid 4 to smithi138138488-28 2023-06-27T12:41:04.686 INFO:tasks.rados.rados.0.smithi138.stdout:614: oid 28 updating version 0 to 113 2023-06-27T12:41:04.687 INFO:tasks.rados.rados.0.smithi138.stdout:614: oid 28 version 113 is already newer than 110 2023-06-27T12:41:04.687 INFO:tasks.rados.rados.0.smithi138.stdout:614: oid 28 version 113 is already newer than 113 2023-06-27T12:41:04.687 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 113 (ObjNum 297 snap 80 seq_num 297) dirty exists 2023-06-27T12:41:04.687 INFO:tasks.rados.rados.0.smithi138.stdout:614: left oid 28 (ObjNum 297 snap 80 seq_num 297) 2023-06-27T12:41:04.687 INFO:tasks.rados.rados.0.smithi138.stdout:616: finishing write tid 1 to smithi138138488-46 2023-06-27T12:41:04.728 INFO:tasks.rados.rados.0.smithi138.stdout:616: finishing write tid 2 to smithi138138488-46 2023-06-27T12:41:04.728 INFO:tasks.rados.rados.0.smithi138.stdout:616: finishing write tid 3 to smithi138138488-46 2023-06-27T12:41:04.728 INFO:tasks.rados.rados.0.smithi138.stdout:616: finishing write tid 4 to smithi138138488-46 2023-06-27T12:41:04.728 INFO:tasks.rados.rados.0.smithi138.stdout:616: finishing write tid 5 to smithi138138488-46 2023-06-27T12:41:04.729 INFO:tasks.rados.rados.0.smithi138.stdout:616: oid 46 updating version 0 to 85 2023-06-27T12:41:04.729 INFO:tasks.rados.rados.0.smithi138.stdout:616: oid 46 version 85 is already newer than 85 2023-06-27T12:41:04.729 INFO:tasks.rados.rados.0.smithi138.stdout:616: oid 46 version 85 is already newer than 85 2023-06-27T12:41:04.729 INFO:tasks.rados.rados.0.smithi138.stdout:616: oid 46 version 85 is already newer than 84 2023-06-27T12:41:04.729 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 85 (ObjNum 299 snap 80 seq_num 299) dirty exists 2023-06-27T12:41:04.729 INFO:tasks.rados.rados.0.smithi138.stdout:616: left oid 46 (ObjNum 299 snap 80 seq_num 299) 2023-06-27T12:41:04.735 INFO:tasks.rados.rados.0.smithi138.stdout:614: done (3 left) 2023-06-27T12:41:04.735 INFO:tasks.rados.rados.0.smithi138.stdout:616: done (2 left) 2023-06-27T12:41:04.735 INFO:tasks.rados.rados.0.smithi138.stdout:618: done (1 left) 2023-06-27T12:41:04.735 INFO:tasks.rados.rados.0.smithi138.stdout:619: write oid 46 current snap is 80 2023-06-27T12:41:04.735 INFO:tasks.rados.rados.0.smithi138.stdout:619: seq_num 302 ranges {675714=462965,1589209=641430,2943979=131735} 2023-06-27T12:41:04.741 INFO:tasks.rados.rados.0.smithi138.stdout:619: writing smithi138138488-46 from 675714 to 1138679 tid 1 2023-06-27T12:41:04.747 INFO:tasks.rados.rados.0.smithi138.stdout:619: writing smithi138138488-46 from 1589209 to 2230639 tid 2 2023-06-27T12:41:04.752 INFO:tasks.rados.rados.0.smithi138.stdout:619: writing smithi138138488-46 from 2943979 to 3075714 tid 3 2023-06-27T12:41:04.752 INFO:tasks.rados.rados.0.smithi138.stdout:620: read oid 34 snap -1 2023-06-27T12:41:04.752 INFO:tasks.rados.rados.0.smithi138.stdout:620: expect (ObjNum 243 snap 67 seq_num 243) 2023-06-27T12:41:04.753 INFO:tasks.rados.rados.0.smithi138.stdout:621: write (excl) oid 39 current snap is 80 2023-06-27T12:41:04.753 INFO:tasks.rados.rados.0.smithi138.stdout:617: finishing write tid 1 to smithi138138488-26 2023-06-27T12:41:04.753 INFO:tasks.rados.rados.0.smithi138.stdout:621: seq_num 303 ranges {692521=538214,1931338=785088} 2023-06-27T12:41:04.759 INFO:tasks.rados.rados.0.smithi138.stdout:621: writing smithi138138488-39 from 692521 to 1230735 tid 1 2023-06-27T12:41:04.768 INFO:tasks.rados.rados.0.smithi138.stdout:621: writing smithi138138488-39 from 1931338 to 2716426 tid 2 2023-06-27T12:41:04.768 INFO:tasks.rados.rados.0.smithi138.stdout:622: read oid 18 snap 78 2023-06-27T12:41:04.768 INFO:tasks.rados.rados.0.smithi138.stdout:622: expect (ObjNum 254 snap 70 seq_num 254) 2023-06-27T12:41:04.768 INFO:tasks.rados.rados.0.smithi138.stdout:623: delete oid 7 current snap is 80 2023-06-27T12:41:04.768 INFO:tasks.rados.rados.0.smithi138.stdout:617: finishing write tid 2 to smithi138138488-26 2023-06-27T12:41:04.769 INFO:tasks.rados.rados.0.smithi138.stdout:617: finishing write tid 3 to smithi138138488-26 2023-06-27T12:41:04.769 INFO:tasks.rados.rados.0.smithi138.stdout:617: finishing write tid 4 to smithi138138488-26 2023-06-27T12:41:04.769 INFO:tasks.rados.rados.0.smithi138.stdout:617: oid 26 updating version 0 to 90 2023-06-27T12:41:04.769 INFO:tasks.rados.rados.0.smithi138.stdout:617: oid 26 version 90 is already newer than 90 2023-06-27T12:41:04.769 INFO:tasks.rados.rados.0.smithi138.stdout:617: oid 26 version 90 is already newer than 90 2023-06-27T12:41:04.769 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 90 (ObjNum 300 snap 80 seq_num 300) dirty exists 2023-06-27T12:41:04.769 INFO:tasks.rados.rados.0.smithi138.stdout:617: left oid 26 (ObjNum 300 snap 80 seq_num 300) 2023-06-27T12:41:04.770 INFO:tasks.rados.rados.0.smithi138.stdout:619: finishing write tid 1 to smithi138138488-46 2023-06-27T12:41:04.788 INFO:tasks.rados.rados.0.smithi138.stdout:620: expect (ObjNum 243 snap 67 seq_num 243) 2023-06-27T12:41:04.915 INFO:tasks.rados.rados.0.smithi138.stdout:617: done (5 left) 2023-06-27T12:41:04.915 INFO:tasks.rados.rados.0.smithi138.stdout:620: done (4 left) 2023-06-27T12:41:04.916 INFO:tasks.rados.rados.0.smithi138.stdout:623: done (3 left) 2023-06-27T12:41:04.916 INFO:tasks.rados.rados.0.smithi138.stdout:624: write oid 29 current snap is 80 2023-06-27T12:41:04.916 INFO:tasks.rados.rados.0.smithi138.stdout:619: finishing write tid 2 to smithi138138488-46 2023-06-27T12:41:04.916 INFO:tasks.rados.rados.0.smithi138.stdout:619: finishing write tid 3 to smithi138138488-46 2023-06-27T12:41:04.917 INFO:tasks.rados.rados.0.smithi138.stdout:624: seq_num 305 ranges {726135=405065,1764655=788757,3026248=99887} 2023-06-27T12:41:04.921 INFO:tasks.rados.rados.0.smithi138.stdout:624: writing smithi138138488-29 from 726135 to 1131200 tid 1 2023-06-27T12:41:04.929 INFO:tasks.rados.rados.0.smithi138.stdout:624: writing smithi138138488-29 from 1764655 to 2553412 tid 2 2023-06-27T12:41:04.932 INFO:tasks.rados.rados.0.smithi138.stdout:624: writing smithi138138488-29 from 3026248 to 3126135 tid 3 2023-06-27T12:41:04.932 INFO:tasks.rados.rados.0.smithi138.stdout:625: read oid 6 snap -1 2023-06-27T12:41:04.932 INFO:tasks.rados.rados.0.smithi138.stdout:625: expect (ObjNum 290 snap 78 seq_num 290) 2023-06-27T12:41:04.933 INFO:tasks.rados.rados.0.smithi138.stdout:626: snap_create 2023-06-27T12:41:04.933 INFO:tasks.rados.rados.0.smithi138.stdout:622: expect (ObjNum 254 snap 70 seq_num 254) 2023-06-27T12:41:05.099 INFO:tasks.rados.rados.0.smithi138.stdout:619: finishing write tid 4 to smithi138138488-46 2023-06-27T12:41:05.099 INFO:tasks.rados.rados.0.smithi138.stdout:619: finishing write tid 5 to smithi138138488-46 2023-06-27T12:41:05.099 INFO:tasks.rados.rados.0.smithi138.stdout:619: oid 46 updating version 0 to 89 2023-06-27T12:41:05.099 INFO:tasks.rados.rados.0.smithi138.stdout:619: oid 46 version 89 is already newer than 87 2023-06-27T12:41:05.099 INFO:tasks.rados.rados.0.smithi138.stdout:619: oid 46 version 89 is already newer than 89 2023-06-27T12:41:05.100 INFO:tasks.rados.rados.0.smithi138.stdout:619: oid 46 version 89 is already newer than 89 2023-06-27T12:41:05.100 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 89 (ObjNum 302 snap 80 seq_num 302) dirty exists 2023-06-27T12:41:05.100 INFO:tasks.rados.rados.0.smithi138.stdout:619: left oid 46 (ObjNum 302 snap 80 seq_num 302) 2023-06-27T12:41:05.100 INFO:tasks.rados.rados.0.smithi138.stdout:621: finishing write tid 1 to smithi138138488-39 2023-06-27T12:41:05.100 INFO:tasks.rados.rados.0.smithi138.stdout:621: finishing write tid 2 to smithi138138488-39 2023-06-27T12:41:05.100 INFO:tasks.rados.rados.0.smithi138.stdout:621: finishing write tid 3 to smithi138138488-39 2023-06-27T12:41:05.100 INFO:tasks.rados.rados.0.smithi138.stdout:621: finishing write tid 4 to smithi138138488-39 2023-06-27T12:41:05.101 INFO:tasks.rados.rados.0.smithi138.stdout:621: oid 39 updating version 0 to 85 2023-06-27T12:41:05.101 INFO:tasks.rados.rados.0.smithi138.stdout:621: oid 39 version 85 is already newer than 85 2023-06-27T12:41:05.101 INFO:tasks.rados.rados.0.smithi138.stdout:621: oid 39 version 85 is already newer than 82 2023-06-27T12:41:05.101 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 39 v 85 (ObjNum 303 snap 80 seq_num 303) dirty exists 2023-06-27T12:41:05.101 INFO:tasks.rados.rados.0.smithi138.stdout:621: left oid 39 (ObjNum 303 snap 80 seq_num 303) 2023-06-27T12:41:05.101 INFO:tasks.rados.rados.0.smithi138.stdout:625: expect (ObjNum 290 snap 78 seq_num 290) 2023-06-27T12:41:05.302 INFO:tasks.rados.rados.0.smithi138.stdout:624: finishing write tid 1 to smithi138138488-29 2023-06-27T12:41:05.302 INFO:tasks.rados.rados.0.smithi138.stdout:624: finishing write tid 2 to smithi138138488-29 2023-06-27T12:41:05.302 INFO:tasks.rados.rados.0.smithi138.stdout:624: finishing write tid 3 to smithi138138488-29 2023-06-27T12:41:05.302 INFO:tasks.rados.rados.0.smithi138.stdout:624: finishing write tid 4 to smithi138138488-29 2023-06-27T12:41:05.302 INFO:tasks.rados.rados.0.smithi138.stdout:624: finishing write tid 5 to smithi138138488-29 2023-06-27T12:41:05.303 INFO:tasks.rados.rados.0.smithi138.stdout:624: oid 29 updating version 0 to 57 2023-06-27T12:41:05.303 INFO:tasks.rados.rados.0.smithi138.stdout:624: oid 29 version 57 is already newer than 57 2023-06-27T12:41:05.303 INFO:tasks.rados.rados.0.smithi138.stdout:624: oid 29 version 57 is already newer than 57 2023-06-27T12:41:05.303 INFO:tasks.rados.rados.0.smithi138.stdout:624: oid 29 version 57 is already newer than 57 2023-06-27T12:41:05.303 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 57 (ObjNum 305 snap 80 seq_num 305) dirty exists 2023-06-27T12:41:05.303 INFO:tasks.rados.rados.0.smithi138.stdout:624: left oid 29 (ObjNum 305 snap 80 seq_num 305) 2023-06-27T12:41:05.408 INFO:tasks.rados.rados.0.smithi138.stdout:619: done (5 left) 2023-06-27T12:41:05.408 INFO:tasks.rados.rados.0.smithi138.stdout:621: done (4 left) 2023-06-27T12:41:05.408 INFO:tasks.rados.rados.0.smithi138.stdout:622: done (3 left) 2023-06-27T12:41:05.408 INFO:tasks.rados.rados.0.smithi138.stdout:624: done (2 left) 2023-06-27T12:41:05.409 INFO:tasks.rados.rados.0.smithi138.stdout:625: done (1 left) 2023-06-27T12:41:05.409 INFO:tasks.rados.rados.0.smithi138.stdout:626: done (0 left) 2023-06-27T12:41:05.409 INFO:tasks.rados.rados.0.smithi138.stdout:627: delete oid 4 current snap is 81 2023-06-27T12:41:05.487 INFO:tasks.rados.rados.0.smithi138.stdout:627: done (0 left) 2023-06-27T12:41:05.487 INFO:tasks.rados.rados.0.smithi138.stdout:628: delete oid 1 current snap is 81 2023-06-27T12:41:05.569 INFO:tasks.rados.rados.0.smithi138.stdout:628: done (0 left) 2023-06-27T12:41:05.570 INFO:tasks.rados.rados.0.smithi138.stdout:629: read oid 41 snap 51 2023-06-27T12:41:05.570 INFO:tasks.rados.rados.0.smithi138.stdout:629: expect (ObjNum 113 snap 10 seq_num 113) 2023-06-27T12:41:05.570 INFO:tasks.rados.rados.0.smithi138.stdout:630: read oid 28 snap -1 2023-06-27T12:41:05.570 INFO:tasks.rados.rados.0.smithi138.stdout:630: expect (ObjNum 297 snap 80 seq_num 297) 2023-06-27T12:41:05.570 INFO:tasks.rados.rados.0.smithi138.stdout:631: snap_remove snap 67 2023-06-27T12:41:05.884 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:06.010 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:06.005+0000 7f3c1a143700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:06.013 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:06.009+0000 7f3c1a143700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:06.057 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:06.059 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:06.060 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:06.491 INFO:tasks.rados.rados.0.smithi138.stdout:631: done (2 left) 2023-06-27T12:41:06.491 INFO:tasks.rados.rados.0.smithi138.stdout:632: delete oid 4 current snap is 81 2023-06-27T12:41:06.491 INFO:tasks.rados.rados.0.smithi138.stdout:629: expect (ObjNum 113 snap 10 seq_num 113) 2023-06-27T12:41:06.702 INFO:tasks.rados.rados.0.smithi138.stdout:630: expect (ObjNum 297 snap 80 seq_num 297) 2023-06-27T12:41:06.892 INFO:tasks.rados.rados.0.smithi138.stdout:629: done (2 left) 2023-06-27T12:41:06.892 INFO:tasks.rados.rados.0.smithi138.stdout:630: done (1 left) 2023-06-27T12:41:06.893 INFO:tasks.rados.rados.0.smithi138.stdout:632: done (0 left) 2023-06-27T12:41:06.893 INFO:tasks.rados.rados.0.smithi138.stdout:633: write oid 44 current snap is 81 2023-06-27T12:41:06.893 INFO:tasks.rados.rados.0.smithi138.stdout:633: seq_num 309 ranges {793363=706061,2282230=796095} 2023-06-27T12:41:06.900 INFO:tasks.rados.rados.0.smithi138.stdout:633: writing smithi138138488-44 from 793363 to 1499424 tid 1 2023-06-27T12:41:06.909 INFO:tasks.rados.rados.0.smithi138.stdout:633: writing smithi138138488-44 from 2282230 to 3078325 tid 2 2023-06-27T12:41:06.909 INFO:tasks.rados.rados.0.smithi138.stdout:634: delete oid 19 current snap is 81 2023-06-27T12:41:06.995 INFO:tasks.rados.rados.0.smithi138.stdout:634: done (1 left) 2023-06-27T12:41:06.996 INFO:tasks.rados.rados.0.smithi138.stdout:635: read oid 42 snap -1 2023-06-27T12:41:06.996 INFO:tasks.rados.rados.0.smithi138.stdout:635: expect deleted 2023-06-27T12:41:06.996 INFO:tasks.rados.rados.0.smithi138.stdout:636: read oid 13 snap -1 2023-06-27T12:41:06.996 INFO:tasks.rados.rados.0.smithi138.stdout:636: expect (ObjNum 248 snap 68 seq_num 248) 2023-06-27T12:41:06.996 INFO:tasks.rados.rados.0.smithi138.stdout:637: read oid 15 snap -1 2023-06-27T12:41:06.996 INFO:tasks.rados.rados.0.smithi138.stdout:637: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:41:06.997 INFO:tasks.rados.rados.0.smithi138.stdout:638: read oid 22 snap -1 2023-06-27T12:41:06.997 INFO:tasks.rados.rados.0.smithi138.stdout:638: expect (ObjNum 275 snap 72 seq_num 275) 2023-06-27T12:41:06.997 INFO:tasks.rados.rados.0.smithi138.stdout:639: write (excl) oid 32 current snap is 81 2023-06-27T12:41:06.997 INFO:tasks.rados.rados.0.smithi138.stdout:639: seq_num 311 ranges {426977=456559,1482841=799764,2958080=268897} 2023-06-27T12:41:07.000 INFO:tasks.rados.rados.0.smithi138.stdout:639: writing smithi138138488-32 from 426977 to 883536 tid 1 2023-06-27T12:41:07.009 INFO:tasks.rados.rados.0.smithi138.stdout:639: writing smithi138138488-32 from 1482841 to 2282605 tid 2 2023-06-27T12:41:07.015 INFO:tasks.rados.rados.0.smithi138.stdout:639: writing smithi138138488-32 from 2958080 to 3226977 tid 3 2023-06-27T12:41:07.015 INFO:tasks.rados.rados.0.smithi138.stdout:640: delete oid 31 current snap is 81 2023-06-27T12:41:07.015 INFO:tasks.rados.rados.0.smithi138.stdout:633: finishing write tid 1 to smithi138138488-44 2023-06-27T12:41:07.015 INFO:tasks.rados.rados.0.smithi138.stdout:633: finishing write tid 2 to smithi138138488-44 2023-06-27T12:41:07.015 INFO:tasks.rados.rados.0.smithi138.stdout:633: finishing write tid 3 to smithi138138488-44 2023-06-27T12:41:07.015 INFO:tasks.rados.rados.0.smithi138.stdout:633: finishing write tid 4 to smithi138138488-44 2023-06-27T12:41:07.016 INFO:tasks.rados.rados.0.smithi138.stdout:633: oid 44 updating version 0 to 208 2023-06-27T12:41:07.016 INFO:tasks.rados.rados.0.smithi138.stdout:633: oid 44 version 208 is already newer than 208 2023-06-27T12:41:07.016 INFO:tasks.rados.rados.0.smithi138.stdout:633: oid 44 version 208 is already newer than 208 2023-06-27T12:41:07.016 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 208 (ObjNum 309 snap 81 seq_num 309) dirty exists 2023-06-27T12:41:07.016 INFO:tasks.rados.rados.0.smithi138.stdout:633: left oid 44 (ObjNum 309 snap 81 seq_num 309) 2023-06-27T12:41:07.026 INFO:tasks.rados.rados.0.smithi138.stdout:633: done (6 left) 2023-06-27T12:41:07.026 INFO:tasks.rados.rados.0.smithi138.stdout:635: done (5 left) 2023-06-27T12:41:07.026 INFO:tasks.rados.rados.0.smithi138.stdout:640: done (4 left) 2023-06-27T12:41:07.026 INFO:tasks.rados.rados.0.smithi138.stdout:641: read oid 40 snap 63 2023-06-27T12:41:07.027 INFO:tasks.rados.rados.0.smithi138.stdout:641: expect (ObjNum 238 snap 62 seq_num 238) 2023-06-27T12:41:07.027 INFO:tasks.rados.rados.0.smithi138.stdout:642: delete oid 48 current snap is 81 2023-06-27T12:41:07.029 INFO:tasks.rados.rados.0.smithi138.stdout:637: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:41:07.157 INFO:tasks.rados.rados.0.smithi138.stdout:641: expect (ObjNum 238 snap 62 seq_num 238) 2023-06-27T12:41:07.275 INFO:tasks.rados.rados.0.smithi138.stdout:638: expect (ObjNum 275 snap 72 seq_num 275) 2023-06-27T12:41:07.424 INFO:tasks.rados.rados.0.smithi138.stdout:636: expect (ObjNum 248 snap 68 seq_num 248) 2023-06-27T12:41:07.532 INFO:tasks.rados.rados.0.smithi138.stdout:639: finishing write tid 1 to smithi138138488-32 2023-06-27T12:41:07.533 INFO:tasks.rados.rados.0.smithi138.stdout:639: finishing write tid 2 to smithi138138488-32 2023-06-27T12:41:07.533 INFO:tasks.rados.rados.0.smithi138.stdout:639: finishing write tid 3 to smithi138138488-32 2023-06-27T12:41:07.533 INFO:tasks.rados.rados.0.smithi138.stdout:639: finishing write tid 4 to smithi138138488-32 2023-06-27T12:41:07.533 INFO:tasks.rados.rados.0.smithi138.stdout:639: finishing write tid 5 to smithi138138488-32 2023-06-27T12:41:07.533 INFO:tasks.rados.rados.0.smithi138.stdout:639: oid 32 updating version 0 to 246 2023-06-27T12:41:07.533 INFO:tasks.rados.rados.0.smithi138.stdout:639: oid 32 version 246 is already newer than 246 2023-06-27T12:41:07.533 INFO:tasks.rados.rados.0.smithi138.stdout:639: oid 32 version 246 is already newer than 242 2023-06-27T12:41:07.534 INFO:tasks.rados.rados.0.smithi138.stdout:639: oid 32 version 246 is already newer than 246 2023-06-27T12:41:07.534 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 246 (ObjNum 311 snap 81 seq_num 311) dirty exists 2023-06-27T12:41:07.534 INFO:tasks.rados.rados.0.smithi138.stdout:639: left oid 32 (ObjNum 311 snap 81 seq_num 311) 2023-06-27T12:41:07.534 INFO:tasks.rados.rados.0.smithi138.stdout:636: done (5 left) 2023-06-27T12:41:07.534 INFO:tasks.rados.rados.0.smithi138.stdout:637: done (4 left) 2023-06-27T12:41:07.534 INFO:tasks.rados.rados.0.smithi138.stdout:638: done (3 left) 2023-06-27T12:41:07.534 INFO:tasks.rados.rados.0.smithi138.stdout:639: done (2 left) 2023-06-27T12:41:07.534 INFO:tasks.rados.rados.0.smithi138.stdout:641: done (1 left) 2023-06-27T12:41:07.535 INFO:tasks.rados.rados.0.smithi138.stdout:642: done (0 left) 2023-06-27T12:41:07.535 INFO:tasks.rados.rados.0.smithi138.stdout:643: snap_remove snap 75 2023-06-27T12:41:08.061 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:08.191 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:08.186+0000 7fd6dfa3f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:08.194 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:08.190+0000 7fd6dfa3f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:08.239 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:08.241 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:08.242 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:08.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~33s 2023-06-27T12:41:08.498 INFO:tasks.rados.rados.0.smithi138.stdout:643: done (0 left) 2023-06-27T12:41:08.498 INFO:tasks.rados.rados.0.smithi138.stdout:644: snap_remove snap 73 2023-06-27T12:41:09.259 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:09.259 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:09.259 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:09.390 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:09.383+0000 7f4e06ef7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:09.400 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:09.389+0000 7f4e06ef7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:09.498 INFO:tasks.rados.rados.0.smithi138.stdout:644: done (0 left) 2023-06-27T12:41:09.499 INFO:tasks.rados.rados.0.smithi138.stdout:645: read oid 39 snap -1 2023-06-27T12:41:09.499 INFO:tasks.rados.rados.0.smithi138.stdout:645: expect (ObjNum 303 snap 80 seq_num 303) 2023-06-27T12:41:09.499 INFO:tasks.rados.rados.0.smithi138.stdout:646: write oid 45 current snap is 81 2023-06-27T12:41:09.499 INFO:tasks.rados.rados.0.smithi138.stdout:646: seq_num 314 ranges {477398=798659,1774640=663444,2873055=404343} 2023-06-27T12:41:09.506 INFO:tasks.rados.rados.0.smithi138.stdout:646: writing smithi138138488-45 from 477398 to 1276057 tid 1 2023-06-27T12:41:09.512 INFO:tasks.rados.rados.0.smithi138.stdout:646: writing smithi138138488-45 from 1774640 to 2438084 tid 2 2023-06-27T12:41:09.518 INFO:tasks.rados.rados.0.smithi138.stdout:646: writing smithi138138488-45 from 2873055 to 3277398 tid 3 2023-06-27T12:41:09.518 INFO:tasks.rados.rados.0.smithi138.stdout:647: delete oid 27 current snap is 81 2023-06-27T12:41:09.559 INFO:tasks.rados.rados.0.smithi138.stdout:645: expect (ObjNum 303 snap 80 seq_num 303) 2023-06-27T12:41:09.612 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:09.614 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:09.626 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2023-06-27T12:41:09.493214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67161333,"num_objects":168,"num_object_clones":127,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":921,"num_read_kb":612412,"num_write":796,"num_write_kb":229699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1681,"ondisk_log_size":1681,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":729880,"kb_used_data":606200,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376757480,"statfs":{"total":386547056640,"available":385799659520,"internally_reserved":0,"allocated":620748800,"data_stored":731754026,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-38019722,"num_objects":-30,"num_object_clones":-13,"num_object_copies":-90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-58,"num_read_kb":-12053,"num_write":-306,"num_write_kb":-51238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550654"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":412,"reported_epoch":176,"state":"active+undersized","last_fresh":"2023-06-27T12:41:08.523704+0000","last_change":"2023-06-27T12:41:08.538016+0000","last_active":"2023-06-27T12:41:08.523704+0000","last_peered":"2023-06-27T12:41:08.523704+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:08.523704+0000","last_undegraded":"2023-06-27T12:41:08.523704+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"146'102","reported_seq":502,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.578443+0000","last_change":"2023-06-27T12:41:08.579345+0000","last_active":"2023-06-27T12:41:08.578443+0000","last_peered":"2023-06-27T12:41:08.578443+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:08.578443+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":40,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680733,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":76,"num_read_kb":85333,"num_write":68,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":16}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":502,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.576321+0000","last_change":"2023-06-27T12:41:08.577406+0000","last_active":"2023-06-27T12:41:08.576321+0000","last_peered":"2023-06-27T12:41:08.576321+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:08.576321+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":40,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":69,"num_read_kb":27101,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"173'69","reported_seq":470,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.608878+0000","last_change":"2023-06-27T12:41:08.610888+0000","last_active":"2023-06-27T12:41:08.608878+0000","last_peered":"2023-06-27T12:41:08.608878+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:08.608878+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":34,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3344593,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":64,"num_read_kb":31561,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":96,"reported_epoch":176,"state":"active+undersized","last_fresh":"2023-06-27T12:41:08.523394+0000","last_change":"2023-06-27T12:41:08.534627+0000","last_active":"2023-06-27T12:41:08.523394+0000","last_peered":"2023-06-27T12:41:08.523394+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:08.523394+0000","last_undegraded":"2023-06-27T12:41:08.523394+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":42,"num_read_kb":18496,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"175'208","reported_seq":534,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-06-27T12:41:08.528255+0000","last_change":"2023-06-27T12:41:08.663066+0000","last_active":"2023-06-27T12:41:08.528255+0000","last_peered":"2023-06-27T12:41:08.528255+0000","last_clean":"2023-06-27T12:41:08.528255+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:08.528255+0000","last_undegraded":"2023-06-27T12:41:08.528255+0000","last_fullsized":"2023-06-27T12:41:08.528255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8235529,"num_objects":27,"num_object_clones":21,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":174,"num_read_kb":118393,"num_write":122,"num_write_kb":35372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"173'115","reported_seq":433,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-06-27T12:41:08.528339+0000","last_change":"2023-06-27T12:41:08.654257+0000","last_active":"2023-06-27T12:41:08.528339+0000","last_peered":"2023-06-27T12:41:08.528339+0000","last_clean":"2023-06-27T12:41:08.528339+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:08.528339+0000","last_undegraded":"2023-06-27T12:41:08.528339+0000","last_fullsized":"2023-06-27T12:41:08.528339+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2991679,"num_objects":16,"num_object_clones":13,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":67,"num_read_kb":37353,"num_write":67,"num_write_kb":19727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":432,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.510554+0000","last_change":"2023-06-27T12:41:08.510879+0000","last_active":"2023-06-27T12:41:08.510554+0000","last_peered":"2023-06-27T12:41:08.510554+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:08.510554+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":32,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2773188,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":6}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":278,"reported_epoch":176,"state":"active+undersized","last_fresh":"2023-06-27T12:41:08.545262+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:08.545262+0000","last_peered":"2023-06-27T12:41:08.545262+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:08.545262+0000","last_undegraded":"2023-06-27T12:41:08.545262+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":407,"reported_epoch":176,"state":"active+undersized","last_fresh":"2023-06-27T12:41:08.501255+0000","last_change":"2023-06-27T12:41:08.529816+0000","last_active":"2023-06-27T12:41:08.501255+0000","last_peered":"2023-06-27T12:41:08.501255+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:08.501255+0000","last_undegraded":"2023-06-27T12:41:08.501255+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":267,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-06-27T12:41:08.530842+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:08.530842+0000","last_peered":"2023-06-27T12:41:08.530842+0000","last_clean":"2023-06-27T12:41:08.530842+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:08.530842+0000","last_undegraded":"2023-06-27T12:41:08.530842+0000","last_fullsized":"2023-06-27T12:41:08.530842+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"173'89","reported_seq":503,"reported_epoch":176,"state":"active+undersized+degraded+snaptrim","last_fresh":"2023-06-27T12:41:08.561959+0000","last_change":"2023-06-27T12:41:08.538924+0000","last_active":"2023-06-27T12:41:08.561959+0000","last_peered":"2023-06-27T12:41:08.561959+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:08.561959+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9395179,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":42,"num_read_kb":34775,"num_write":64,"num_write_kb":17475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":270,"reported_epoch":176,"state":"active+undersized","last_fresh":"2023-06-27T12:41:08.502207+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:08.502207+0000","last_peered":"2023-06-27T12:41:08.502207+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:08.502207+0000","last_undegraded":"2023-06-27T12:41:08.502207+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"175'246","reported_seq":84,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.563391+0000","last_change":"2023-06-27T12:41:08.564110+0000","last_active":"2023-06-27T12:41:08.563391+0000","last_peered":"2023-06-27T12:41:08.563391+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:08.563391+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":33,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3109361,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":4709,"num_write":14,"num_write_kb":3807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":3}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":267,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-06-27T12:41:08.506743+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:08.506743+0000","last_peered":"2023-06-27T12:41:08.506743+0000","last_clean":"2023-06-27T12:41:08.506743+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:08.506743+0000","last_undegraded":"2023-06-27T12:41:08.506743+0000","last_fullsized":"2023-06-27T12:41:08.506743+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":19,"reported_epoch":176,"state":"active+undersized","last_fresh":"2023-06-27T12:41:08.532557+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:08.532557+0000","last_peered":"2023-06-27T12:41:08.532557+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:08.532557+0000","last_undegraded":"2023-06-27T12:41:08.532557+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"175'184","reported_seq":558,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.534872+0000","last_change":"2023-06-27T12:41:08.537252+0000","last_active":"2023-06-27T12:41:08.534872+0000","last_peered":"2023-06-27T12:41:08.534872+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:08.534872+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":50,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8655671,"num_objects":27,"num_object_clones":21,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":150,"num_read_kb":77248,"num_write":131,"num_write_kb":37214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":27}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":267,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-06-27T12:41:08.525585+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:08.525585+0000","last_peered":"2023-06-27T12:41:08.525585+0000","last_clean":"2023-06-27T12:41:08.525585+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:08.525585+0000","last_undegraded":"2023-06-27T12:41:08.525585+0000","last_fullsized":"2023-06-27T12:41:08.525585+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'87","reported_seq":111,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.629419+0000","last_change":"2023-06-27T12:41:08.659693+0000","last_active":"2023-06-27T12:41:08.629419+0000","last_peered":"2023-06-27T12:41:08.629419+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:08.629419+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":38,"num_read_kb":22078,"num_write":13,"num_write_kb":4215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":176,"state":"active+undersized","last_fresh":"2023-06-27T12:41:08.538496+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:08.538496+0000","last_peered":"2023-06-27T12:41:08.538496+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:08.538496+0000","last_undegraded":"2023-06-27T12:41:08.538496+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"173'90","reported_seq":525,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.561128+0000","last_change":"2023-06-27T12:41:08.638983+0000","last_active":"2023-06-27T12:41:08.561128+0000","last_peered":"2023-06-27T12:41:08.561128+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:08.561128+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12806967,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":58637,"num_write":72,"num_write_kb":20999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":13}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"173'57","reported_seq":72,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.543412+0000","last_change":"2023-06-27T12:41:08.543780+0000","last_active":"2023-06-27T12:41:08.543412+0000","last_peered":"2023-06-27T12:41:08.543412+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:08.543412+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":33,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218491,"num_objects":1,"num_object_clones":1,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":9230,"num_write":9,"num_write_kb":2650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":278,"reported_epoch":176,"state":"active+undersized","last_fresh":"2023-06-27T12:41:08.534949+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:08.534949+0000","last_peered":"2023-06-27T12:41:08.534949+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:08.534949+0000","last_undegraded":"2023-06-27T12:41:08.534949+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":267,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-06-27T12:41:08.506078+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:08.506078+0000","last_peered":"2023-06-27T12:41:08.506078+0000","last_clean":"2023-06-27T12:41:08.506078+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:08.506078+0000","last_undegraded":"2023-06-27T12:41:08.506078+0000","last_fullsized":"2023-06-27T12:41:08.506078+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":503,"reported_epoch":176,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:08.549973+0000","last_change":"2023-06-27T12:41:08.550426+0000","last_active":"2023-06-27T12:41:08.549973+0000","last_peered":"2023-06-27T12:41:08.549973+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:08.549973+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":16,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":58,"num_read_kb":53464,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":67161333,"num_objects":168,"num_object_clones":127,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":921,"num_read_kb":612412,"num_write":796,"num_write_kb":229699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1584,"ondisk_log_size":1584,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673155,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":170248,"kb_used_data":139328,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94201592,"statfs":{"total":96636764160,"available":96462430208,"internally_reserved":0,"allocated":142671872,"data_stored":169247651,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705860,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":180968,"kb_used_data":150048,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190872,"statfs":{"total":96636764160,"available":96451452928,"internally_reserved":0,"allocated":153649152,"data_stored":180839095,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673155,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":197384,"kb_used_data":166464,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94174456,"statfs":{"total":96636764160,"available":96434642944,"internally_reserved":0,"allocated":170459136,"data_stored":204627728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:09.628 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.d 3.c 3.3 3.2 3.4 3.7 2023-06-27T12:41:09.730 INFO:tasks.rados.rados.0.smithi138.stdout:646: finishing write tid 1 to smithi138138488-45 2023-06-27T12:41:09.731 INFO:tasks.rados.rados.0.smithi138.stdout:646: finishing write tid 2 to smithi138138488-45 2023-06-27T12:41:09.731 INFO:tasks.rados.rados.0.smithi138.stdout:646: finishing write tid 3 to smithi138138488-45 2023-06-27T12:41:09.731 INFO:tasks.rados.rados.0.smithi138.stdout:646: finishing write tid 4 to smithi138138488-45 2023-06-27T12:41:09.731 INFO:tasks.rados.rados.0.smithi138.stdout:646: finishing write tid 5 to smithi138138488-45 2023-06-27T12:41:09.731 INFO:tasks.rados.rados.0.smithi138.stdout:646: oid 45 updating version 0 to 251 2023-06-27T12:41:09.731 INFO:tasks.rados.rados.0.smithi138.stdout:646: oid 45 version 251 is already newer than 251 2023-06-27T12:41:09.731 INFO:tasks.rados.rados.0.smithi138.stdout:646: oid 45 version 251 is already newer than 251 2023-06-27T12:41:09.732 INFO:tasks.rados.rados.0.smithi138.stdout:646: oid 45 version 251 is already newer than 251 2023-06-27T12:41:09.732 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 251 (ObjNum 314 snap 81 seq_num 314) dirty exists 2023-06-27T12:41:09.732 INFO:tasks.rados.rados.0.smithi138.stdout:646: left oid 45 (ObjNum 314 snap 81 seq_num 314) 2023-06-27T12:41:09.732 INFO:tasks.rados.rados.0.smithi138.stdout:645: done (2 left) 2023-06-27T12:41:09.732 INFO:tasks.rados.rados.0.smithi138.stdout:646: done (1 left) 2023-06-27T12:41:09.732 INFO:tasks.rados.rados.0.smithi138.stdout:647: done (0 left) 2023-06-27T12:41:09.732 INFO:tasks.rados.rados.0.smithi138.stdout:648: snap_remove snap 79 2023-06-27T12:41:09.763 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:09.750+0000 7faf865fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:09.766 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:09.762+0000 7faf865fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:09.991 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c] on osd.0 to force-recovery; instructing pg(s) [3.2,3.7,3.d] on osd.1 to force-recovery; instructing pg(s) [3.3,3.4] on osd.2 to force-recovery; 2023-06-27T12:41:09.991 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:41:10.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:10.368 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:10.363+0000 7f2cce597700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:10.371 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:10.367+0000 7f2cce597700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:10.410 INFO:tasks.rados.rados.0.smithi138.stdout:648: done (0 left) 2023-06-27T12:41:10.410 INFO:tasks.rados.rados.0.smithi138.stdout:649: write oid 19 current snap is 81 2023-06-27T12:41:10.411 INFO:tasks.rados.rados.0.smithi138.stdout:649: seq_num 316 ranges {511012=549157,1775251=550760,3095195=215817} 2023-06-27T12:41:10.416 INFO:tasks.rados.rados.0.smithi138.stdout:649: writing smithi138138488-19 from 511012 to 1060169 tid 1 2023-06-27T12:41:10.417 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:10.421 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:10.422 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:10.424 INFO:tasks.rados.rados.0.smithi138.stdout:649: writing smithi138138488-19 from 1775251 to 2326011 tid 2 2023-06-27T12:41:10.430 INFO:tasks.rados.rados.0.smithi138.stdout:649: writing smithi138138488-19 from 3095195 to 3311012 tid 3 2023-06-27T12:41:10.430 INFO:tasks.rados.rados.0.smithi138.stdout:650: delete oid 25 current snap is 81 2023-06-27T12:41:10.492 INFO:tasks.rados.rados.0.smithi138.stdout:649: finishing write tid 1 to smithi138138488-19 2023-06-27T12:41:10.520 INFO:tasks.rados.rados.0.smithi138.stdout:649: finishing write tid 2 to smithi138138488-19 2023-06-27T12:41:10.522 INFO:tasks.rados.rados.0.smithi138.stdout:649: finishing write tid 3 to smithi138138488-19 2023-06-27T12:41:10.526 INFO:tasks.rados.rados.0.smithi138.stdout:649: finishing write tid 4 to smithi138138488-19 2023-06-27T12:41:10.526 INFO:tasks.rados.rados.0.smithi138.stdout:650: done (1 left) 2023-06-27T12:41:10.526 INFO:tasks.rados.rados.0.smithi138.stdout:651: read oid 17 snap -1 2023-06-27T12:41:10.527 INFO:tasks.rados.rados.0.smithi138.stdout:651: expect (ObjNum 219 snap 49 seq_num 219) 2023-06-27T12:41:10.527 INFO:tasks.rados.rados.0.smithi138.stdout:652: delete oid 13 current snap is 81 2023-06-27T12:41:10.527 INFO:tasks.rados.rados.0.smithi138.stdout:649: finishing write tid 5 to smithi138138488-19 2023-06-27T12:41:10.527 INFO:tasks.rados.rados.0.smithi138.stdout:649: oid 19 updating version 0 to 188 2023-06-27T12:41:10.528 INFO:tasks.rados.rados.0.smithi138.stdout:649: oid 19 version 188 is already newer than 188 2023-06-27T12:41:10.528 INFO:tasks.rados.rados.0.smithi138.stdout:649: oid 19 version 188 is already newer than 187 2023-06-27T12:41:10.528 INFO:tasks.rados.rados.0.smithi138.stdout:649: oid 19 version 188 is already newer than 188 2023-06-27T12:41:10.528 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 188 (ObjNum 316 snap 81 seq_num 316) dirty exists 2023-06-27T12:41:10.528 INFO:tasks.rados.rados.0.smithi138.stdout:649: left oid 19 (ObjNum 316 snap 81 seq_num 316) 2023-06-27T12:41:10.596 INFO:tasks.rados.rados.0.smithi138.stdout:649: done (2 left) 2023-06-27T12:41:10.597 INFO:tasks.rados.rados.0.smithi138.stdout:652: done (1 left) 2023-06-27T12:41:10.597 INFO:tasks.rados.rados.0.smithi138.stdout:653: read oid 13 snap -1 2023-06-27T12:41:10.597 INFO:tasks.rados.rados.0.smithi138.stdout:653: expect deleted 2023-06-27T12:41:10.597 INFO:tasks.rados.rados.0.smithi138.stdout:654: delete oid 16 current snap is 81 2023-06-27T12:41:10.611 INFO:tasks.rados.rados.0.smithi138.stdout:651: expect (ObjNum 219 snap 49 seq_num 219) 2023-06-27T12:41:10.889 INFO:tasks.rados.rados.0.smithi138.stdout:651: done (2 left) 2023-06-27T12:41:10.889 INFO:tasks.rados.rados.0.smithi138.stdout:654: done (1 left) 2023-06-27T12:41:10.889 INFO:tasks.rados.rados.0.smithi138.stdout:655: read oid 38 snap -1 2023-06-27T12:41:10.889 INFO:tasks.rados.rados.0.smithi138.stdout:655: expect deleted 2023-06-27T12:41:10.889 INFO:tasks.rados.rados.0.smithi138.stdout:656: snap_remove snap 63 2023-06-27T12:41:11.512 INFO:tasks.rados.rados.0.smithi138.stdout:656: done (2 left) 2023-06-27T12:41:11.512 INFO:tasks.rados.rados.0.smithi138.stdout:657: read oid 40 snap -1 2023-06-27T12:41:11.512 INFO:tasks.rados.rados.0.smithi138.stdout:657: expect deleted 2023-06-27T12:41:11.512 INFO:tasks.rados.rados.0.smithi138.stdout:658: write oid 24 current snap is 81 2023-06-27T12:41:11.513 INFO:tasks.rados.rados.0.smithi138.stdout:658: seq_num 320 ranges {578240=566506,1609179=558098,2804887=551750} 2023-06-27T12:41:11.518 INFO:tasks.rados.rados.0.smithi138.stdout:658: writing smithi138138488-24 from 578240 to 1144746 tid 1 2023-06-27T12:41:11.524 INFO:tasks.rados.rados.0.smithi138.stdout:658: writing smithi138138488-24 from 1609179 to 2167277 tid 2 2023-06-27T12:41:11.531 INFO:tasks.rados.rados.0.smithi138.stdout:658: writing smithi138138488-24 from 2804887 to 3356637 tid 3 2023-06-27T12:41:11.531 INFO:tasks.rados.rados.0.smithi138.stdout:659: read oid 15 snap -1 2023-06-27T12:41:11.531 INFO:tasks.rados.rados.0.smithi138.stdout:659: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:41:11.531 INFO:tasks.rados.rados.0.smithi138.stdout:660: read oid 28 snap -1 2023-06-27T12:41:11.532 INFO:tasks.rados.rados.0.smithi138.stdout:660: expect (ObjNum 297 snap 80 seq_num 297) 2023-06-27T12:41:11.532 INFO:tasks.rados.rados.0.smithi138.stdout:661: write (excl) oid 17 current snap is 81 2023-06-27T12:41:11.532 INFO:tasks.rados.rados.0.smithi138.stdout:661: seq_num 321 ranges {595047=641755,1667661=418109,2548663=763022} 2023-06-27T12:41:11.538 INFO:tasks.rados.rados.0.smithi138.stdout:661: writing smithi138138488-17 from 595047 to 1236802 tid 1 2023-06-27T12:41:11.543 INFO:tasks.rados.rados.0.smithi138.stdout:661: writing smithi138138488-17 from 1667661 to 2085770 tid 2 2023-06-27T12:41:11.551 INFO:tasks.rados.rados.0.smithi138.stdout:661: writing smithi138138488-17 from 2548663 to 3311685 tid 3 2023-06-27T12:41:11.551 INFO:tasks.rados.rados.0.smithi138.stdout:662: read oid 6 snap -1 2023-06-27T12:41:11.551 INFO:tasks.rados.rados.0.smithi138.stdout:662: expect (ObjNum 290 snap 78 seq_num 290) 2023-06-27T12:41:11.551 INFO:tasks.rados.rados.0.smithi138.stdout:663: write (excl) oid 45 current snap is 81 2023-06-27T12:41:11.552 INFO:tasks.rados.rados.0.smithi138.stdout:663: seq_num 322 ranges {611854=717004,2126143=561767,3259733=152121} 2023-06-27T12:41:11.558 INFO:tasks.rados.rados.0.smithi138.stdout:663: writing smithi138138488-45 from 611854 to 1328858 tid 1 2023-06-27T12:41:11.566 INFO:tasks.rados.rados.0.smithi138.stdout:663: writing smithi138138488-45 from 2126143 to 2687910 tid 2 2023-06-27T12:41:11.570 INFO:tasks.rados.rados.0.smithi138.stdout:663: writing smithi138138488-45 from 3259733 to 3411854 tid 3 2023-06-27T12:41:11.570 INFO:tasks.rados.rados.0.smithi138.stdout:664: snap_remove snap 76 2023-06-27T12:41:12.422 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:12.516 INFO:tasks.rados.rados.0.smithi138.stdout:664: done (9 left) 2023-06-27T12:41:12.517 INFO:tasks.rados.rados.0.smithi138.stdout:665: delete oid 31 current snap is 81 2023-06-27T12:41:12.517 INFO:tasks.rados.rados.0.smithi138.stdout:658: finishing write tid 1 to smithi138138488-24 2023-06-27T12:41:12.517 INFO:tasks.rados.rados.0.smithi138.stdout:658: finishing write tid 2 to smithi138138488-24 2023-06-27T12:41:12.517 INFO:tasks.rados.rados.0.smithi138.stdout:661: finishing write tid 1 to smithi138138488-17 2023-06-27T12:41:12.518 INFO:tasks.rados.rados.0.smithi138.stdout:659: expect (ObjNum 196 snap 40 seq_num 196) 2023-06-27T12:41:12.570 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:12.565+0000 7f425fe88700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:12.574 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:12.571+0000 7f425fe88700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:12.632 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:12.635 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:12.635 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:12.653 INFO:tasks.rados.rados.0.smithi138.stdout:663: finishing write tid 1 to smithi138138488-45 2023-06-27T12:41:12.653 INFO:tasks.rados.rados.0.smithi138.stdout:662: expect (ObjNum 290 snap 78 seq_num 290) 2023-06-27T12:41:12.857 INFO:tasks.rados.rados.0.smithi138.stdout:661: finishing write tid 2 to smithi138138488-17 2023-06-27T12:41:12.858 INFO:tasks.rados.rados.0.smithi138.stdout:653: done (9 left) 2023-06-27T12:41:12.858 INFO:tasks.rados.rados.0.smithi138.stdout:655: done (8 left) 2023-06-27T12:41:12.858 INFO:tasks.rados.rados.0.smithi138.stdout:657: done (7 left) 2023-06-27T12:41:12.858 INFO:tasks.rados.rados.0.smithi138.stdout:659: done (6 left) 2023-06-27T12:41:12.858 INFO:tasks.rados.rados.0.smithi138.stdout:662: done (5 left) 2023-06-27T12:41:12.858 INFO:tasks.rados.rados.0.smithi138.stdout:665: done (4 left) 2023-06-27T12:41:12.858 INFO:tasks.rados.rados.0.smithi138.stdout:666: delete oid 38 current snap is 81 2023-06-27T12:41:12.859 INFO:tasks.rados.rados.0.smithi138.stdout:660: expect (ObjNum 297 snap 80 seq_num 297) 2023-06-27T12:41:13.050 INFO:tasks.rados.rados.0.smithi138.stdout:658: finishing write tid 3 to smithi138138488-24 2023-06-27T12:41:13.051 INFO:tasks.rados.rados.0.smithi138.stdout:661: finishing write tid 3 to smithi138138488-17 2023-06-27T12:41:13.051 INFO:tasks.rados.rados.0.smithi138.stdout:661: finishing write tid 4 to smithi138138488-17 2023-06-27T12:41:13.051 INFO:tasks.rados.rados.0.smithi138.stdout:661: finishing write tid 5 to smithi138138488-17 2023-06-27T12:41:13.051 INFO:tasks.rados.rados.0.smithi138.stdout:661: oid 17 updating version 0 to 104 2023-06-27T12:41:13.052 INFO:tasks.rados.rados.0.smithi138.stdout:661: oid 17 updating version 104 to 108 2023-06-27T12:41:13.052 INFO:tasks.rados.rados.0.smithi138.stdout:661: oid 17 version 108 is already newer than 108 2023-06-27T12:41:13.052 INFO:tasks.rados.rados.0.smithi138.stdout:661: oid 17 version 108 is already newer than 108 2023-06-27T12:41:13.052 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 108 (ObjNum 321 snap 81 seq_num 321) dirty exists 2023-06-27T12:41:13.052 INFO:tasks.rados.rados.0.smithi138.stdout:661: left oid 17 (ObjNum 321 snap 81 seq_num 321) 2023-06-27T12:41:13.053 INFO:tasks.rados.rados.0.smithi138.stdout:658: finishing write tid 4 to smithi138138488-24 2023-06-27T12:41:13.053 INFO:tasks.rados.rados.0.smithi138.stdout:658: finishing write tid 5 to smithi138138488-24 2023-06-27T12:41:13.053 INFO:tasks.rados.rados.0.smithi138.stdout:658: oid 24 updating version 0 to 233 2023-06-27T12:41:13.054 INFO:tasks.rados.rados.0.smithi138.stdout:658: oid 24 version 233 is already newer than 233 2023-06-27T12:41:13.054 INFO:tasks.rados.rados.0.smithi138.stdout:658: oid 24 version 233 is already newer than 233 2023-06-27T12:41:13.054 INFO:tasks.rados.rados.0.smithi138.stdout:658: oid 24 version 233 is already newer than 233 2023-06-27T12:41:13.054 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 233 (ObjNum 320 snap 81 seq_num 320) dirty exists 2023-06-27T12:41:13.054 INFO:tasks.rados.rados.0.smithi138.stdout:658: left oid 24 (ObjNum 320 snap 81 seq_num 320) 2023-06-27T12:41:13.055 INFO:tasks.rados.rados.0.smithi138.stdout:663: finishing write tid 2 to smithi138138488-45 2023-06-27T12:41:13.055 INFO:tasks.rados.rados.0.smithi138.stdout:663: finishing write tid 3 to smithi138138488-45 2023-06-27T12:41:13.055 INFO:tasks.rados.rados.0.smithi138.stdout:658: done (4 left) 2023-06-27T12:41:13.055 INFO:tasks.rados.rados.0.smithi138.stdout:660: done (3 left) 2023-06-27T12:41:13.055 INFO:tasks.rados.rados.0.smithi138.stdout:661: done (2 left) 2023-06-27T12:41:13.056 INFO:tasks.rados.rados.0.smithi138.stdout:666: done (1 left) 2023-06-27T12:41:13.056 INFO:tasks.rados.rados.0.smithi138.stdout:667: read oid 26 snap -1 2023-06-27T12:41:13.056 INFO:tasks.rados.rados.0.smithi138.stdout:667: expect (ObjNum 300 snap 80 seq_num 300) 2023-06-27T12:41:13.056 INFO:tasks.rados.rados.0.smithi138.stdout:668: read oid 23 snap -1 2023-06-27T12:41:13.056 INFO:tasks.rados.rados.0.smithi138.stdout:668: expect (ObjNum 205 snap 44 seq_num 205) 2023-06-27T12:41:13.057 INFO:tasks.rados.rados.0.smithi138.stdout:669: read oid 42 snap -1 2023-06-27T12:41:13.057 INFO:tasks.rados.rados.0.smithi138.stdout:669: expect deleted 2023-06-27T12:41:13.057 INFO:tasks.rados.rados.0.smithi138.stdout:670: delete oid 10 current snap is 81 2023-06-27T12:41:13.057 INFO:tasks.rados.rados.0.smithi138.stdout:663: finishing write tid 4 to smithi138138488-45 2023-06-27T12:41:13.057 INFO:tasks.rados.rados.0.smithi138.stdout:663: finishing write tid 5 to smithi138138488-45 2023-06-27T12:41:13.057 INFO:tasks.rados.rados.0.smithi138.stdout:663: oid 45 updating version 0 to 257 2023-06-27T12:41:13.057 INFO:tasks.rados.rados.0.smithi138.stdout:663: oid 45 version 257 is already newer than 257 2023-06-27T12:41:13.058 INFO:tasks.rados.rados.0.smithi138.stdout:663: oid 45 version 257 is already newer than 257 2023-06-27T12:41:13.058 INFO:tasks.rados.rados.0.smithi138.stdout:663: oid 45 version 257 is already newer than 254 2023-06-27T12:41:13.058 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 257 (ObjNum 322 snap 81 seq_num 322) dirty exists 2023-06-27T12:41:13.058 INFO:tasks.rados.rados.0.smithi138.stdout:663: left oid 45 (ObjNum 322 snap 81 seq_num 322) 2023-06-27T12:41:13.063 INFO:tasks.rados.rados.0.smithi138.stdout:663: done (4 left) 2023-06-27T12:41:13.063 INFO:tasks.rados.rados.0.smithi138.stdout:669: done (3 left) 2023-06-27T12:41:13.063 INFO:tasks.rados.rados.0.smithi138.stdout:670: done (2 left) 2023-06-27T12:41:13.063 INFO:tasks.rados.rados.0.smithi138.stdout:671: read oid 38 snap 80 2023-06-27T12:41:13.064 INFO:tasks.rados.rados.0.smithi138.stdout:671: expect deleted 2023-06-27T12:41:13.064 INFO:tasks.rados.rados.0.smithi138.stdout:672: delete oid 12 current snap is 81 2023-06-27T12:41:13.065 INFO:tasks.rados.rados.0.smithi138.stdout:672: done (3 left) 2023-06-27T12:41:13.065 INFO:tasks.rados.rados.0.smithi138.stdout:673: read oid 29 snap -1 2023-06-27T12:41:13.065 INFO:tasks.rados.rados.0.smithi138.stdout:673: expect (ObjNum 305 snap 80 seq_num 305) 2023-06-27T12:41:13.065 INFO:tasks.rados.rados.0.smithi138.stdout:674: delete oid 50 current snap is 81 2023-06-27T12:41:13.085 INFO:tasks.rados.rados.0.smithi138.stdout:673: expect (ObjNum 305 snap 80 seq_num 305) 2023-06-27T12:41:13.274 INFO:tasks.rados.rados.0.smithi138.stdout:667: expect (ObjNum 300 snap 80 seq_num 300) 2023-06-27T12:41:13.482 INFO:tasks.rados.rados.0.smithi138.stdout:668: expect (ObjNum 205 snap 44 seq_num 205) 2023-06-27T12:41:13.612 INFO:tasks.rados.rados.0.smithi138.stdout:667: done (4 left) 2023-06-27T12:41:13.612 INFO:tasks.rados.rados.0.smithi138.stdout:668: done (3 left) 2023-06-27T12:41:13.612 INFO:tasks.rados.rados.0.smithi138.stdout:671: done (2 left) 2023-06-27T12:41:13.612 INFO:tasks.rados.rados.0.smithi138.stdout:673: done (1 left) 2023-06-27T12:41:13.613 INFO:tasks.rados.rados.0.smithi138.stdout:674: done (0 left) 2023-06-27T12:41:13.613 INFO:tasks.rados.rados.0.smithi138.stdout:675: snap_remove snap 51 2023-06-27T12:41:13.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~39s 2023-06-27T12:41:14.522 INFO:tasks.rados.rados.0.smithi138.stdout:675: done (0 left) 2023-06-27T12:41:14.522 INFO:tasks.rados.rados.0.smithi138.stdout:676: snap_create 2023-06-27T12:41:14.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:14.774 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:14.769+0000 7f1e05b0b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:14.777 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:14.773+0000 7f1e05b0b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:14.823 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:14.826 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:14.827 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:15.003 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:15.003 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:15.004 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:15.125+0000 7f2e3d42b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:15.135 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:15.131+0000 7f2e3d42b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:15.328 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:15.330 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:15.341 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-06-27T12:41:14.517509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":66169720,"num_objects":174,"num_object_clones":133,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":21,"num_read":971,"num_read_kb":666028,"num_write":821,"num_write_kb":237628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1729,"ondisk_log_size":1729,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":746544,"kb_used_data":621892,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376740816,"statfs":{"total":386547056640,"available":385782595584,"internally_reserved":0,"allocated":636817408,"data_stored":753193276,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-991613,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":50,"num_read_kb":53616,"num_write":25,"num_write_kb":7929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.573720"},"pg_stats":[{"pgid":"3.f","version":"48'6","reported_seq":424,"reported_epoch":180,"state":"active+undersized","last_fresh":"2023-06-27T12:41:12.536834+0000","last_change":"2023-06-27T12:41:12.539884+0000","last_active":"2023-06-27T12:41:12.536834+0000","last_peered":"2023-06-27T12:41:12.536834+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:12.536834+0000","last_undegraded":"2023-06-27T12:41:12.536834+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"179'108","reported_seq":538,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:13.120912+0000","last_change":"2023-06-27T12:41:13.121282+0000","last_active":"2023-06-27T12:41:13.120912+0000","last_peered":"2023-06-27T12:41:13.120912+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:13.120912+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":29,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395047,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":85,"num_read_kb":96119,"num_write":73,"num_write_kb":21776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":527,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:12.594292+0000","last_change":"2023-06-27T12:41:12.594292+0000","last_active":"2023-06-27T12:41:12.594292+0000","last_peered":"2023-06-27T12:41:12.594292+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:12.594292+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":29,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":69,"num_read_kb":27101,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"173'69","reported_seq":491,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:12.582019+0000","last_change":"2023-06-27T12:41:12.582907+0000","last_active":"2023-06-27T12:41:12.582019+0000","last_peered":"2023-06-27T12:41:12.582019+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:12.582019+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":29,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3344593,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":64,"num_read_kb":31561,"num_write":51,"num_write_kb":14073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":111,"reported_epoch":180,"state":"active+undersized","last_fresh":"2023-06-27T12:41:12.552713+0000","last_change":"2023-06-27T12:41:12.565019+0000","last_active":"2023-06-27T12:41:12.552713+0000","last_peered":"2023-06-27T12:41:12.552713+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:12.552713+0000","last_undegraded":"2023-06-27T12:41:12.552713+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":24671,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"179'233","reported_seq":555,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-06-27T12:41:12.566903+0000","last_change":"2023-06-27T12:41:12.566903+0000","last_active":"2023-06-27T12:41:12.566903+0000","last_peered":"2023-06-27T12:41:12.566903+0000","last_clean":"2023-06-27T12:41:12.566903+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:12.566903+0000","last_undegraded":"2023-06-27T12:41:12.566903+0000","last_fullsized":"2023-06-27T12:41:12.566903+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8655704,"num_objects":28,"num_object_clones":22,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":177,"num_read_kb":118394,"num_write":127,"num_write_kb":37011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"173'115","reported_seq":452,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-06-27T12:41:12.569515+0000","last_change":"2023-06-27T12:41:12.569515+0000","last_active":"2023-06-27T12:41:12.569515+0000","last_peered":"2023-06-27T12:41:12.569515+0000","last_clean":"2023-06-27T12:41:12.569515+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:12.569515+0000","last_undegraded":"2023-06-27T12:41:12.569515+0000","last_fullsized":"2023-06-27T12:41:12.569515+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2991679,"num_objects":16,"num_object_clones":13,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":74,"num_read_kb":44587,"num_write":67,"num_write_kb":19727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"165'46","reported_seq":449,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:12.581399+0000","last_change":"2023-06-27T12:41:12.581399+0000","last_active":"2023-06-27T12:41:12.581399+0000","last_peered":"2023-06-27T12:41:12.581399+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:12.581399+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":29,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":180,"state":"active+undersized","last_fresh":"2023-06-27T12:41:13.065749+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:13.065749+0000","last_peered":"2023-06-27T12:41:13.065749+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:13.065749+0000","last_undegraded":"2023-06-27T12:41:13.065749+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":417,"reported_epoch":180,"state":"active+undersized","last_fresh":"2023-06-27T12:41:12.567915+0000","last_change":"2023-06-27T12:41:12.583289+0000","last_active":"2023-06-27T12:41:12.567915+0000","last_peered":"2023-06-27T12:41:12.567915+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:12.567915+0000","last_undegraded":"2023-06-27T12:41:12.567915+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":275,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-06-27T12:41:12.531617+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:12.531617+0000","last_peered":"2023-06-27T12:41:12.531617+0000","last_clean":"2023-06-27T12:41:12.531617+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:12.531617+0000","last_undegraded":"2023-06-27T12:41:12.531617+0000","last_fullsized":"2023-06-27T12:41:12.531617+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"173'89","reported_seq":525,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:13.103652+0000","last_change":"2023-06-27T12:41:13.097496+0000","last_active":"2023-06-27T12:41:13.103652+0000","last_peered":"2023-06-27T12:41:13.103652+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:13.103652+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9395179,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":46,"num_read_kb":44042,"num_write":64,"num_write_kb":17475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":276,"reported_epoch":180,"state":"active+undersized","last_fresh":"2023-06-27T12:41:12.552892+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:12.552892+0000","last_peered":"2023-06-27T12:41:12.552892+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:12.552892+0000","last_undegraded":"2023-06-27T12:41:12.552892+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"179'257","reported_seq":112,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:13.096025+0000","last_change":"2023-06-27T12:41:13.100497+0000","last_active":"2023-06-27T12:41:13.096025+0000","last_peered":"2023-06-27T12:41:13.096025+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:13.096025+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":32,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663926,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":13,"num_read_kb":4711,"num_write":24,"num_write_kb":7029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":5}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":273,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-06-27T12:41:12.552480+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:12.552480+0000","last_peered":"2023-06-27T12:41:12.552480+0000","last_clean":"2023-06-27T12:41:12.552480+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:12.552480+0000","last_undegraded":"2023-06-27T12:41:12.552480+0000","last_fullsized":"2023-06-27T12:41:12.552480+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":27,"reported_epoch":180,"state":"active+undersized","last_fresh":"2023-06-27T12:41:12.548976+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:12.548976+0000","last_peered":"2023-06-27T12:41:12.548976+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:12.548976+0000","last_undegraded":"2023-06-27T12:41:12.548976+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"178'188","reported_seq":582,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:12.568865+0000","last_change":"2023-06-27T12:41:12.569223+0000","last_active":"2023-06-27T12:41:12.568865+0000","last_peered":"2023-06-27T12:41:12.568865+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:12.568865+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":44,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11966683,"num_objects":27,"num_object_clones":21,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":151,"num_read_kb":77249,"num_write":136,"num_write_kb":38500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":27}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":275,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-06-27T12:41:12.544571+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:12.544571+0000","last_peered":"2023-06-27T12:41:12.544571+0000","last_clean":"2023-06-27T12:41:12.544571+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:12.544571+0000","last_undegraded":"2023-06-27T12:41:12.544571+0000","last_fullsized":"2023-06-27T12:41:12.544571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'87","reported_seq":125,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:12.581498+0000","last_change":"2023-06-27T12:41:12.582435+0000","last_active":"2023-06-27T12:41:12.581498+0000","last_peered":"2023-06-27T12:41:12.581498+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:12.581498+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":45,"num_read_kb":29957,"num_write":13,"num_write_kb":4215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":180,"state":"active+undersized","last_fresh":"2023-06-27T12:41:13.062609+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:13.062609+0000","last_peered":"2023-06-27T12:41:13.062609+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:13.062609+0000","last_undegraded":"2023-06-27T12:41:13.062609+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"173'90","reported_seq":541,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:13.085771+0000","last_change":"2023-06-27T12:41:13.072557+0000","last_active":"2023-06-27T12:41:13.085771+0000","last_peered":"2023-06-27T12:41:13.085771+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:13.085771+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12806967,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":63128,"num_write":72,"num_write_kb":20999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":13}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"178'59","reported_seq":89,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:13.077032+0000","last_change":"2023-06-27T12:41:12.557687+0000","last_active":"2023-06-27T12:41:13.077032+0000","last_peered":"2023-06-27T12:41:13.077032+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:13.077032+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":33,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":29,"num_read_kb":17010,"num_write":9,"num_write_kb":2650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":286,"reported_epoch":180,"state":"active+undersized","last_fresh":"2023-06-27T12:41:13.058818+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:13.058818+0000","last_peered":"2023-06-27T12:41:13.058818+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:13.058818+0000","last_undegraded":"2023-06-27T12:41:13.058818+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":273,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-06-27T12:41:12.556221+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:12.556221+0000","last_peered":"2023-06-27T12:41:12.556221+0000","last_clean":"2023-06-27T12:41:12.556221+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:12.556221+0000","last_undegraded":"2023-06-27T12:41:12.556221+0000","last_fullsized":"2023-06-27T12:41:12.556221+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":521,"reported_epoch":180,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:13.101762+0000","last_change":"2023-06-27T12:41:13.102215+0000","last_active":"2023-06-27T12:41:13.101762+0000","last_peered":"2023-06-27T12:41:13.101762+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:13.101762+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":14,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":58,"num_read_kb":53464,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":66169720,"num_objects":174,"num_object_clones":133,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":21,"num_read":971,"num_read_kb":666028,"num_write":821,"num_write_kb":237628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1632,"ondisk_log_size":1632,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":175964,"kb_used_data":145044,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94195876,"statfs":{"total":96636764160,"available":96456577024,"internally_reserved":0,"allocated":148525056,"data_stored":177224653,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":183932,"kb_used_data":153012,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94187908,"statfs":{"total":96636764160,"available":96448417792,"internally_reserved":0,"allocated":156684288,"data_stored":185008607,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":205368,"kb_used_data":173476,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94166472,"statfs":{"total":96636764160,"available":96426467328,"internally_reserved":0,"allocated":177639424,"data_stored":213920464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:15.343 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.2 3.6 2023-06-27T12:41:15.413 INFO:tasks.rados.rados.0.smithi138.stdout:676: done (0 left) 2023-06-27T12:41:15.414 INFO:tasks.rados.rados.0.smithi138.stdout:677: read oid 9 snap -1 2023-06-27T12:41:15.414 INFO:tasks.rados.rados.0.smithi138.stdout:677: expect (ObjNum 270 snap 70 seq_num 270) 2023-06-27T12:41:15.414 INFO:tasks.rados.rados.0.smithi138.stdout:678: delete oid 16 current snap is 82 2023-06-27T12:41:15.474 INFO:tasks.rados.rados.0.smithi138.stdout:678: done (1 left) 2023-06-27T12:41:15.474 INFO:tasks.rados.rados.0.smithi138.stdout:679: read oid 18 snap 80 2023-06-27T12:41:15.474 INFO:tasks.rados.rados.0.smithi138.stdout:679: expect (ObjNum 254 snap 70 seq_num 254) 2023-06-27T12:41:15.475 INFO:tasks.rados.rados.0.smithi138.stdout:680: read oid 16 snap 77 2023-06-27T12:41:15.475 INFO:tasks.rados.rados.0.smithi138.stdout:680: expect deleted 2023-06-27T12:41:15.475 INFO:tasks.rados.rados.0.smithi138.stdout:681: snap_remove snap 58 2023-06-27T12:41:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:15.469+0000 7f08927fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:15.478+0000 7f08927fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:15.683 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.2] on osd.1 to force-backfill; instructing pg(s) [3.6] on osd.2 to force-backfill; 2023-06-27T12:41:15.683 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:41:16.418 INFO:tasks.rados.rados.0.smithi138.stdout:681: done (3 left) 2023-06-27T12:41:16.419 INFO:tasks.rados.rados.0.smithi138.stdout:682: snap_create 2023-06-27T12:41:16.419 INFO:tasks.rados.rados.0.smithi138.stdout:677: expect (ObjNum 270 snap 70 seq_num 270) 2023-06-27T12:41:16.650 INFO:tasks.rados.rados.0.smithi138.stdout:679: expect (ObjNum 254 snap 70 seq_num 254) 2023-06-27T12:41:16.828 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:16.955 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:16.950+0000 7f0440547700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:16.958 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:16.954+0000 7f0440547700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:17.003 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:17.005 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:17.006 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:17.420 INFO:tasks.rados.rados.0.smithi138.stdout:677: done (3 left) 2023-06-27T12:41:17.421 INFO:tasks.rados.rados.0.smithi138.stdout:679: done (2 left) 2023-06-27T12:41:17.421 INFO:tasks.rados.rados.0.smithi138.stdout:680: done (1 left) 2023-06-27T12:41:17.421 INFO:tasks.rados.rados.0.smithi138.stdout:682: done (0 left) 2023-06-27T12:41:17.421 INFO:tasks.rados.rados.0.smithi138.stdout:683: read oid 46 snap -1 2023-06-27T12:41:17.422 INFO:tasks.rados.rados.0.smithi138.stdout:683: expect (ObjNum 302 snap 80 seq_num 302) 2023-06-27T12:41:17.422 INFO:tasks.rados.rados.0.smithi138.stdout:684: write (excl) oid 9 current snap is 83 2023-06-27T12:41:17.422 INFO:tasks.rados.rados.0.smithi138.stdout:684: seq_num 329 ranges {729503=560100,2019164=716432,3335341=194162} 2023-06-27T12:41:17.427 INFO:tasks.rados.rados.0.smithi138.stdout:684: writing smithi138138488-9 from 729503 to 1289603 tid 1 2023-06-27T12:41:17.435 INFO:tasks.rados.rados.0.smithi138.stdout:684: writing smithi138138488-9 from 2019164 to 2735596 tid 2 2023-06-27T12:41:17.439 INFO:tasks.rados.rados.0.smithi138.stdout:684: writing smithi138138488-9 from 3335341 to 3529503 tid 3 2023-06-27T12:41:17.439 INFO:tasks.rados.rados.0.smithi138.stdout:685: write oid 28 current snap is 83 2023-06-27T12:41:17.440 INFO:tasks.rados.rados.0.smithi138.stdout:685: seq_num 330 ranges {746310=635349,2077646=576443,3362764=183546} 2023-06-27T12:41:17.448 INFO:tasks.rados.rados.0.smithi138.stdout:685: writing smithi138138488-28 from 746310 to 1381659 tid 1 2023-06-27T12:41:17.457 INFO:tasks.rados.rados.0.smithi138.stdout:685: writing smithi138138488-28 from 2077646 to 2654089 tid 2 2023-06-27T12:41:17.463 INFO:tasks.rados.rados.0.smithi138.stdout:685: writing smithi138138488-28 from 3362764 to 3546310 tid 3 2023-06-27T12:41:17.464 INFO:tasks.rados.rados.0.smithi138.stdout:686: write (excl) oid 33 current snap is 83 2023-06-27T12:41:17.464 INFO:tasks.rados.rados.0.smithi138.stdout:686: seq_num 331 ranges {763117=710598,2136128=720101,3390187=172930} 2023-06-27T12:41:17.471 INFO:tasks.rados.rados.0.smithi138.stdout:686: writing smithi138138488-33 from 763117 to 1473715 tid 1 2023-06-27T12:41:17.480 INFO:tasks.rados.rados.0.smithi138.stdout:686: writing smithi138138488-33 from 2136128 to 2856229 tid 2 2023-06-27T12:41:17.485 INFO:tasks.rados.rados.0.smithi138.stdout:686: writing smithi138138488-33 from 3390187 to 3563117 tid 3 2023-06-27T12:41:17.485 INFO:tasks.rados.rados.0.smithi138.stdout:687: delete oid 24 current snap is 83 2023-06-27T12:41:17.485 INFO:tasks.rados.rados.0.smithi138.stdout:683: expect (ObjNum 302 snap 80 seq_num 302) 2023-06-27T12:41:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:684: finishing write tid 1 to smithi138138488-9 2023-06-27T12:41:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:685: finishing write tid 1 to smithi138138488-28 2023-06-27T12:41:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:685: finishing write tid 2 to smithi138138488-28 2023-06-27T12:41:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:685: finishing write tid 3 to smithi138138488-28 2023-06-27T12:41:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:686: finishing write tid 1 to smithi138138488-33 2023-06-27T12:41:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:685: finishing write tid 4 to smithi138138488-28 2023-06-27T12:41:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:685: finishing write tid 5 to smithi138138488-28 2023-06-27T12:41:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:685: oid 28 updating version 0 to 134 2023-06-27T12:41:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:685: oid 28 version 134 is already newer than 134 2023-06-27T12:41:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:685: oid 28 version 134 is already newer than 134 2023-06-27T12:41:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:685: oid 28 version 134 is already newer than 134 2023-06-27T12:41:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 134 (ObjNum 330 snap 83 seq_num 330) dirty exists 2023-06-27T12:41:17.695 INFO:tasks.rados.rados.0.smithi138.stdout:685: left oid 28 (ObjNum 330 snap 83 seq_num 330) 2023-06-27T12:41:17.695 INFO:tasks.rados.rados.0.smithi138.stdout:686: finishing write tid 2 to smithi138138488-33 2023-06-27T12:41:17.695 INFO:tasks.rados.rados.0.smithi138.stdout:683: done (4 left) 2023-06-27T12:41:17.695 INFO:tasks.rados.rados.0.smithi138.stdout:685: done (3 left) 2023-06-27T12:41:17.695 INFO:tasks.rados.rados.0.smithi138.stdout:687: done (2 left) 2023-06-27T12:41:17.695 INFO:tasks.rados.rados.0.smithi138.stdout:688: write (excl) oid 19 current snap is 83 2023-06-27T12:41:17.695 INFO:tasks.rados.rados.0.smithi138.stdout:684: finishing write tid 2 to smithi138138488-9 2023-06-27T12:41:17.695 INFO:tasks.rados.rados.0.smithi138.stdout:688: seq_num 333 ranges {796731=461096,1736739=723770,2928680=668051} 2023-06-27T12:41:17.699 INFO:tasks.rados.rados.0.smithi138.stdout:688: writing smithi138138488-19 from 796731 to 1257827 tid 1 2023-06-27T12:41:17.706 INFO:tasks.rados.rados.0.smithi138.stdout:688: writing smithi138138488-19 from 1736739 to 2460509 tid 2 2023-06-27T12:41:17.714 INFO:tasks.rados.rados.0.smithi138.stdout:688: writing smithi138138488-19 from 2928680 to 3596731 tid 3 2023-06-27T12:41:17.715 INFO:tasks.rados.rados.0.smithi138.stdout:689: delete oid 29 current snap is 83 2023-06-27T12:41:17.715 INFO:tasks.rados.rados.0.smithi138.stdout:684: finishing write tid 3 to smithi138138488-9 2023-06-27T12:41:17.715 INFO:tasks.rados.rados.0.smithi138.stdout:686: finishing write tid 3 to smithi138138488-33 2023-06-27T12:41:17.715 INFO:tasks.rados.rados.0.smithi138.stdout:684: finishing write tid 4 to smithi138138488-9 2023-06-27T12:41:17.715 INFO:tasks.rados.rados.0.smithi138.stdout:686: finishing write tid 4 to smithi138138488-33 2023-06-27T12:41:17.715 INFO:tasks.rados.rados.0.smithi138.stdout:684: finishing write tid 5 to smithi138138488-9 2023-06-27T12:41:17.715 INFO:tasks.rados.rados.0.smithi138.stdout:684: oid 9 updating version 0 to 193 2023-06-27T12:41:17.715 INFO:tasks.rados.rados.0.smithi138.stdout:684: oid 9 version 193 is already newer than 193 2023-06-27T12:41:17.716 INFO:tasks.rados.rados.0.smithi138.stdout:684: oid 9 version 193 is already newer than 193 2023-06-27T12:41:17.716 INFO:tasks.rados.rados.0.smithi138.stdout:684: oid 9 version 193 is already newer than 189 2023-06-27T12:41:17.716 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 193 (ObjNum 329 snap 83 seq_num 329) dirty exists 2023-06-27T12:41:17.716 INFO:tasks.rados.rados.0.smithi138.stdout:684: left oid 9 (ObjNum 329 snap 83 seq_num 329) 2023-06-27T12:41:17.716 INFO:tasks.rados.rados.0.smithi138.stdout:686: finishing write tid 5 to smithi138138488-33 2023-06-27T12:41:17.716 INFO:tasks.rados.rados.0.smithi138.stdout:686: oid 33 updating version 0 to 74 2023-06-27T12:41:17.716 INFO:tasks.rados.rados.0.smithi138.stdout:686: oid 33 version 74 is already newer than 74 2023-06-27T12:41:17.717 INFO:tasks.rados.rados.0.smithi138.stdout:686: oid 33 version 74 is already newer than 74 2023-06-27T12:41:17.717 INFO:tasks.rados.rados.0.smithi138.stdout:686: oid 33 version 74 is already newer than 70 2023-06-27T12:41:17.717 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 74 (ObjNum 331 snap 83 seq_num 331) dirty exists 2023-06-27T12:41:17.717 INFO:tasks.rados.rados.0.smithi138.stdout:686: left oid 33 (ObjNum 331 snap 83 seq_num 331) 2023-06-27T12:41:17.734 INFO:tasks.rados.rados.0.smithi138.stdout:688: finishing write tid 1 to smithi138138488-19 2023-06-27T12:41:17.749 INFO:tasks.rados.rados.0.smithi138.stdout:684: done (3 left) 2023-06-27T12:41:17.749 INFO:tasks.rados.rados.0.smithi138.stdout:686: done (2 left) 2023-06-27T12:41:17.749 INFO:tasks.rados.rados.0.smithi138.stdout:689: done (1 left) 2023-06-27T12:41:17.750 INFO:tasks.rados.rados.0.smithi138.stdout:690: snap_remove snap 80 2023-06-27T12:41:18.534 INFO:tasks.rados.rados.0.smithi138.stdout:690: done (1 left) 2023-06-27T12:41:18.534 INFO:tasks.rados.rados.0.smithi138.stdout:691: write oid 3 current snap is 83 2023-06-27T12:41:18.534 INFO:tasks.rados.rados.0.smithi138.stdout:688: finishing write tid 2 to smithi138138488-19 2023-06-27T12:41:18.535 INFO:tasks.rados.rados.0.smithi138.stdout:691: seq_num 335 ranges {430345=727947,1570056=727439,2699879=469889} 2023-06-27T12:41:18.540 INFO:tasks.rados.rados.0.smithi138.stdout:691: writing smithi138138488-3 from 430345 to 1158292 tid 1 2023-06-27T12:41:18.546 INFO:tasks.rados.rados.0.smithi138.stdout:691: writing smithi138138488-3 from 1570056 to 2297495 tid 2 2023-06-27T12:41:18.551 INFO:tasks.rados.rados.0.smithi138.stdout:691: writing smithi138138488-3 from 2699879 to 3169768 tid 3 2023-06-27T12:41:18.551 INFO:tasks.rados.rados.0.smithi138.stdout:692: snap_remove snap 77 2023-06-27T12:41:19.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:19.141 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:19.136+0000 7fbf553fb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:19.140+0000 7fbf553fb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:19.190 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:19.193 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:19.194 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:19.337 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~44s 2023-06-27T12:41:19.534 INFO:tasks.rados.rados.0.smithi138.stdout:692: done (2 left) 2023-06-27T12:41:19.535 INFO:tasks.rados.rados.0.smithi138.stdout:693: snap_create 2023-06-27T12:41:19.535 INFO:tasks.rados.rados.0.smithi138.stdout:688: finishing write tid 3 to smithi138138488-19 2023-06-27T12:41:19.535 INFO:tasks.rados.rados.0.smithi138.stdout:688: finishing write tid 4 to smithi138138488-19 2023-06-27T12:41:19.535 INFO:tasks.rados.rados.0.smithi138.stdout:688: finishing write tid 5 to smithi138138488-19 2023-06-27T12:41:19.536 INFO:tasks.rados.rados.0.smithi138.stdout:688: oid 19 updating version 0 to 194 2023-06-27T12:41:19.536 INFO:tasks.rados.rados.0.smithi138.stdout:688: oid 19 updating version 194 to 198 2023-06-27T12:41:19.536 INFO:tasks.rados.rados.0.smithi138.stdout:688: oid 19 version 198 is already newer than 198 2023-06-27T12:41:19.536 INFO:tasks.rados.rados.0.smithi138.stdout:688: oid 19 version 198 is already newer than 198 2023-06-27T12:41:19.536 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 198 (ObjNum 333 snap 83 seq_num 333) dirty exists 2023-06-27T12:41:19.536 INFO:tasks.rados.rados.0.smithi138.stdout:688: left oid 19 (ObjNum 333 snap 83 seq_num 333) 2023-06-27T12:41:19.536 INFO:tasks.rados.rados.0.smithi138.stdout:691: finishing write tid 1 to smithi138138488-3 2023-06-27T12:41:19.536 INFO:tasks.rados.rados.0.smithi138.stdout:691: finishing write tid 2 to smithi138138488-3 2023-06-27T12:41:19.537 INFO:tasks.rados.rados.0.smithi138.stdout:691: finishing write tid 3 to smithi138138488-3 2023-06-27T12:41:19.537 INFO:tasks.rados.rados.0.smithi138.stdout:691: finishing write tid 4 to smithi138138488-3 2023-06-27T12:41:19.537 INFO:tasks.rados.rados.0.smithi138.stdout:691: finishing write tid 5 to smithi138138488-3 2023-06-27T12:41:19.537 INFO:tasks.rados.rados.0.smithi138.stdout:691: oid 3 updating version 0 to 103 2023-06-27T12:41:19.537 INFO:tasks.rados.rados.0.smithi138.stdout:691: oid 3 version 103 is already newer than 101 2023-06-27T12:41:19.537 INFO:tasks.rados.rados.0.smithi138.stdout:691: oid 3 version 103 is already newer than 102 2023-06-27T12:41:19.537 INFO:tasks.rados.rados.0.smithi138.stdout:691: oid 3 version 103 is already newer than 103 2023-06-27T12:41:19.538 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 103 (ObjNum 335 snap 83 seq_num 335) dirty exists 2023-06-27T12:41:19.538 INFO:tasks.rados.rados.0.smithi138.stdout:691: left oid 3 (ObjNum 335 snap 83 seq_num 335) 2023-06-27T12:41:20.539 INFO:tasks.rados.rados.0.smithi138.stdout:688: done (2 left) 2023-06-27T12:41:20.540 INFO:tasks.rados.rados.0.smithi138.stdout:691: done (1 left) 2023-06-27T12:41:20.540 INFO:tasks.rados.rados.0.smithi138.stdout:693: done (0 left) 2023-06-27T12:41:20.540 INFO:tasks.rados.rados.0.smithi138.stdout:694: read oid 19 snap -1 2023-06-27T12:41:20.540 INFO:tasks.rados.rados.0.smithi138.stdout:694: expect (ObjNum 333 snap 83 seq_num 333) 2023-06-27T12:41:20.540 INFO:tasks.rados.rados.0.smithi138.stdout:695: read oid 30 snap -1 2023-06-27T12:41:20.541 INFO:tasks.rados.rados.0.smithi138.stdout:695: expect (ObjNum 288 snap 78 seq_num 288) 2023-06-27T12:41:20.541 INFO:tasks.rados.rados.0.smithi138.stdout:696: delete oid 42 current snap is 84 2023-06-27T12:41:20.619 INFO:tasks.rados.rados.0.smithi138.stdout:696: done (2 left) 2023-06-27T12:41:20.619 INFO:tasks.rados.rados.0.smithi138.stdout:697: read oid 23 snap 81 2023-06-27T12:41:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:697: expect (ObjNum 205 snap 44 seq_num 205) 2023-06-27T12:41:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:698: read oid 16 snap -1 2023-06-27T12:41:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:698: expect deleted 2023-06-27T12:41:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:699: read oid 26 snap -1 2023-06-27T12:41:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:699: expect (ObjNum 300 snap 80 seq_num 300) 2023-06-27T12:41:20.620 INFO:tasks.rados.rados.0.smithi138.stdout:700: read oid 47 snap -1 2023-06-27T12:41:20.621 INFO:tasks.rados.rados.0.smithi138.stdout:700: expect deleted 2023-06-27T12:41:20.621 INFO:tasks.rados.rados.0.smithi138.stdout:701: snap_remove snap 82 2023-06-27T12:41:20.695 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:20.696 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:20.696 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:20.818+0000 7fa02b9dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:20.828 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:20.825+0000 7fa02b9dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:21.023 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:21.077 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:21.077 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-06-27T12:41:20.534452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":64942875,"num_objects":181,"num_object_clones":140,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":997,"num_read_kb":681580,"num_write":846,"num_write_kb":245688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1787,"ondisk_log_size":1787,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":758372,"kb_used_data":634692,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376728988,"statfs":{"total":386547056640,"available":385770483712,"internally_reserved":0,"allocated":649924608,"data_stored":770780193,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1226845,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":15552,"num_write":25,"num_write_kb":8060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.120832"},"pg_stats":[{"pgid":"3.f","version":"180'7","reported_seq":432,"reported_epoch":184,"state":"active+undersized","last_fresh":"2023-06-27T12:41:17.424516+0000","last_change":"2023-06-27T12:41:16.428545+0000","last_active":"2023-06-27T12:41:17.424516+0000","last_peered":"2023-06-27T12:41:17.424516+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:17.424516+0000","last_undegraded":"2023-06-27T12:41:17.424516+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"179'108","reported_seq":565,"reported_epoch":185,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:18.604200+0000","last_change":"2023-06-27T12:41:18.604761+0000","last_active":"2023-06-27T12:41:18.604200+0000","last_peered":"2023-06-27T12:41:18.604200+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:18.604200+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":20,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395047,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":85,"num_read_kb":96119,"num_write":73,"num_write_kb":21776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":543,"reported_epoch":184,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:17.502388+0000","last_change":"2023-06-27T12:41:17.503544+0000","last_active":"2023-06-27T12:41:17.502388+0000","last_peered":"2023-06-27T12:41:17.502388+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:17.502388+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":27,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":31658,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"184'74","reported_seq":512,"reported_epoch":184,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:17.611030+0000","last_change":"2023-06-27T12:41:17.500287+0000","last_active":"2023-06-27T12:41:17.611030+0000","last_peered":"2023-06-27T12:41:17.611030+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:17.611030+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":24,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563117,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":66,"num_read_kb":31562,"num_write":56,"num_write_kb":15640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":119,"reported_epoch":184,"state":"active+undersized","last_fresh":"2023-06-27T12:41:17.448268+0000","last_change":"2023-06-27T12:41:16.439841+0000","last_active":"2023-06-27T12:41:17.448268+0000","last_peered":"2023-06-27T12:41:17.448268+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:17.448268+0000","last_undegraded":"2023-06-27T12:41:17.448268+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":24671,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"179'233","reported_seq":567,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-06-27T12:41:17.422959+0000","last_change":"2023-06-27T12:41:16.444381+0000","last_active":"2023-06-27T12:41:17.422959+0000","last_peered":"2023-06-27T12:41:17.422959+0000","last_clean":"2023-06-27T12:41:17.422959+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:17.422959+0000","last_undegraded":"2023-06-27T12:41:17.422959+0000","last_fullsized":"2023-06-27T12:41:17.422959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":29,"num_object_clones":23,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":178,"num_read_kb":118394,"num_write":127,"num_write_kb":37011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"184'134","reported_seq":467,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-06-27T12:41:17.561441+0000","last_change":"2023-06-27T12:41:16.434971+0000","last_active":"2023-06-27T12:41:17.561441+0000","last_peered":"2023-06-27T12:41:17.561441+0000","last_clean":"2023-06-27T12:41:17.561441+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:17.561441+0000","last_undegraded":"2023-06-27T12:41:17.561441+0000","last_fullsized":"2023-06-27T12:41:17.561441+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3546310,"num_objects":17,"num_object_clones":14,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":75,"num_read_kb":44588,"num_write":72,"num_write_kb":21091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"180'48","reported_seq":465,"reported_epoch":184,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:17.453506+0000","last_change":"2023-06-27T12:41:17.453848+0000","last_active":"2023-06-27T12:41:17.453506+0000","last_peered":"2023-06-27T12:41:17.453506+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:17.453506+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":24,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":44,"num_read_kb":34033,"num_write":27,"num_write_kb":8474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":296,"reported_epoch":185,"state":"active+undersized","last_fresh":"2023-06-27T12:41:18.541838+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:18.541838+0000","last_peered":"2023-06-27T12:41:18.541838+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:18.541838+0000","last_undegraded":"2023-06-27T12:41:18.541838+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":425,"reported_epoch":184,"state":"active+undersized","last_fresh":"2023-06-27T12:41:17.445771+0000","last_change":"2023-06-27T12:41:16.572402+0000","last_active":"2023-06-27T12:41:17.445771+0000","last_peered":"2023-06-27T12:41:17.445771+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:17.445771+0000","last_undegraded":"2023-06-27T12:41:17.445771+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":281,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-06-27T12:41:17.446970+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:17.446970+0000","last_peered":"2023-06-27T12:41:17.446970+0000","last_clean":"2023-06-27T12:41:17.446970+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:17.446970+0000","last_undegraded":"2023-06-27T12:41:17.446970+0000","last_fullsized":"2023-06-27T12:41:17.446970+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"173'89","reported_seq":542,"reported_epoch":185,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:18.546002+0000","last_change":"2023-06-27T12:41:18.558915+0000","last_active":"2023-06-27T12:41:18.546002+0000","last_peered":"2023-06-27T12:41:18.546002+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:18.546002+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9395179,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":48,"num_read_kb":48519,"num_write":64,"num_write_kb":17475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":282,"reported_epoch":184,"state":"active+undersized","last_fresh":"2023-06-27T12:41:17.435238+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:17.435238+0000","last_peered":"2023-06-27T12:41:17.435238+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:17.435238+0000","last_undegraded":"2023-06-27T12:41:17.435238+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"179'257","reported_seq":142,"reported_epoch":185,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:18.607089+0000","last_change":"2023-06-27T12:41:18.607969+0000","last_active":"2023-06-27T12:41:18.607089+0000","last_peered":"2023-06-27T12:41:18.607089+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:18.607089+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":20,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663926,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":13,"num_read_kb":4711,"num_write":24,"num_write_kb":7029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":5}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":279,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-06-27T12:41:17.440906+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:17.440906+0000","last_peered":"2023-06-27T12:41:17.440906+0000","last_clean":"2023-06-27T12:41:17.440906+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:17.440906+0000","last_undegraded":"2023-06-27T12:41:17.440906+0000","last_fullsized":"2023-06-27T12:41:17.440906+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":33,"reported_epoch":184,"state":"active+undersized","last_fresh":"2023-06-27T12:41:17.446699+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:17.446699+0000","last_peered":"2023-06-27T12:41:17.446699+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:17.446699+0000","last_undegraded":"2023-06-27T12:41:17.446699+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"184'198","reported_seq":610,"reported_epoch":184,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:17.737718+0000","last_change":"2023-06-27T12:41:17.477051+0000","last_active":"2023-06-27T12:41:17.737718+0000","last_peered":"2023-06-27T12:41:17.737718+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:17.737718+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":40,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13244015,"num_objects":29,"num_object_clones":23,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":162,"num_read_kb":83764,"num_write":146,"num_write_kb":41748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":29}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-06-27T12:41:17.441818+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:17.441818+0000","last_peered":"2023-06-27T12:41:17.441818+0000","last_clean":"2023-06-27T12:41:17.441818+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:17.441818+0000","last_undegraded":"2023-06-27T12:41:17.441818+0000","last_fullsized":"2023-06-27T12:41:17.441818+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'87","reported_seq":136,"reported_epoch":184,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:17.438072+0000","last_change":"2023-06-27T12:41:16.579236+0000","last_active":"2023-06-27T12:41:17.438072+0000","last_peered":"2023-06-27T12:41:17.438072+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:17.438072+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":45,"num_read_kb":29957,"num_write":13,"num_write_kb":4215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":185,"state":"active+undersized","last_fresh":"2023-06-27T12:41:18.546810+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:18.546810+0000","last_peered":"2023-06-27T12:41:18.546810+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:18.546810+0000","last_undegraded":"2023-06-27T12:41:18.546810+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"185'103","reported_seq":561,"reported_epoch":185,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:18.605898+0000","last_change":"2023-06-27T12:41:18.556638+0000","last_active":"2023-06-27T12:41:18.605898+0000","last_peered":"2023-06-27T12:41:18.605898+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:18.605898+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12907875,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":63129,"num_write":77,"num_write_kb":22880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"184'67","reported_seq":131,"reported_epoch":184,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:17.721968+0000","last_change":"2023-06-27T12:41:17.520434+0000","last_active":"2023-06-27T12:41:17.721968+0000","last_peered":"2023-06-27T12:41:17.721968+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:17.721968+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":17010,"num_write":9,"num_write_kb":2650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":296,"reported_epoch":185,"state":"active+undersized","last_fresh":"2023-06-27T12:41:18.543259+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:18.543259+0000","last_peered":"2023-06-27T12:41:18.543259+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:18.543259+0000","last_undegraded":"2023-06-27T12:41:18.543259+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":279,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-06-27T12:41:17.443401+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:17.443401+0000","last_peered":"2023-06-27T12:41:17.443401+0000","last_clean":"2023-06-27T12:41:17.443401+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:17.443401+0000","last_undegraded":"2023-06-27T12:41:17.443401+0000","last_fullsized":"2023-06-27T12:41:17.443401+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":550,"reported_epoch":185,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:18.560277+0000","last_change":"2023-06-27T12:41:18.562211+0000","last_active":"2023-06-27T12:41:18.560277+0000","last_peered":"2023-06-27T12:41:18.560277+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:18.560277+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":58,"num_read_kb":53464,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":64942875,"num_objects":181,"num_object_clones":140,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":997,"num_read_kb":681580,"num_write":846,"num_write_kb":245688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1690,"ondisk_log_size":1690,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":179152,"kb_used_data":148232,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94192688,"statfs":{"total":96636764160,"available":96453312512,"internally_reserved":0,"allocated":151789568,"data_stored":181889471,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":191300,"kb_used_data":160380,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94180540,"statfs":{"total":96636764160,"available":96440872960,"internally_reserved":0,"allocated":164229120,"data_stored":195553560,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":206640,"kb_used_data":175720,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94165200,"statfs":{"total":96636764160,"available":96425164800,"internally_reserved":0,"allocated":179937280,"data_stored":216297610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:21.195 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:21.320 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:21.315+0000 7f5c3e7c3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:21.323 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:21.319+0000 7f5c3e7c3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:21.369 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:21.371 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:21.372 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:21.545 INFO:tasks.rados.rados.0.smithi138.stdout:701: done (6 left) 2023-06-27T12:41:21.545 INFO:tasks.rados.rados.0.smithi138.stdout:702: read oid 32 snap 78 2023-06-27T12:41:21.546 INFO:tasks.rados.rados.0.smithi138.stdout:702: expect (ObjNum 287 snap 78 seq_num 287) 2023-06-27T12:41:21.546 INFO:tasks.rados.rados.0.smithi138.stdout:703: delete oid 46 current snap is 84 2023-06-27T12:41:21.546 INFO:tasks.rados.rados.0.smithi138.stdout:694: expect (ObjNum 333 snap 83 seq_num 333) 2023-06-27T12:41:21.698 INFO:tasks.rados.rados.0.smithi138.stdout:695: expect (ObjNum 288 snap 78 seq_num 288) 2023-06-27T12:41:21.802 INFO:tasks.rados.rados.0.smithi138.stdout:699: expect (ObjNum 300 snap 80 seq_num 300) 2023-06-27T12:41:22.024 INFO:tasks.rados.rados.0.smithi138.stdout:697: expect (ObjNum 205 snap 44 seq_num 205) 2023-06-27T12:41:22.153 INFO:tasks.rados.rados.0.smithi138.stdout:702: expect (ObjNum 287 snap 78 seq_num 287) 2023-06-27T12:41:22.293 INFO:tasks.rados.rados.0.smithi138.stdout:694: done (7 left) 2023-06-27T12:41:22.294 INFO:tasks.rados.rados.0.smithi138.stdout:695: done (6 left) 2023-06-27T12:41:22.294 INFO:tasks.rados.rados.0.smithi138.stdout:697: done (5 left) 2023-06-27T12:41:22.294 INFO:tasks.rados.rados.0.smithi138.stdout:698: done (4 left) 2023-06-27T12:41:22.294 INFO:tasks.rados.rados.0.smithi138.stdout:699: done (3 left) 2023-06-27T12:41:22.295 INFO:tasks.rados.rados.0.smithi138.stdout:700: done (2 left) 2023-06-27T12:41:22.295 INFO:tasks.rados.rados.0.smithi138.stdout:702: done (1 left) 2023-06-27T12:41:22.295 INFO:tasks.rados.rados.0.smithi138.stdout:703: done (0 left) 2023-06-27T12:41:22.295 INFO:tasks.rados.rados.0.smithi138.stdout:704: snap_remove snap 78 2023-06-27T12:41:22.553 INFO:tasks.rados.rados.0.smithi138.stdout:704: done (0 left) 2023-06-27T12:41:22.553 INFO:tasks.rados.rados.0.smithi138.stdout:705: write oid 13 current snap is 84 2023-06-27T12:41:22.553 INFO:tasks.rados.rados.0.smithi138.stdout:705: seq_num 338 ranges {480766=553694,1629149=591119,2782148=420058} 2023-06-27T12:41:22.561 INFO:tasks.rados.rados.0.smithi138.stdout:705: writing smithi138138488-13 from 480766 to 1034460 tid 1 2023-06-27T12:41:22.570 INFO:tasks.rados.rados.0.smithi138.stdout:705: writing smithi138138488-13 from 1629149 to 2220268 tid 2 2023-06-27T12:41:22.577 INFO:tasks.rados.rados.0.smithi138.stdout:705: writing smithi138138488-13 from 2782148 to 3202206 tid 3 2023-06-27T12:41:22.578 INFO:tasks.rados.rados.0.smithi138.stdout:706: read oid 49 snap 81 2023-06-27T12:41:22.578 INFO:tasks.rados.rados.0.smithi138.stdout:706: expect (ObjNum 245 snap 68 seq_num 245) 2023-06-27T12:41:22.578 INFO:tasks.rados.rados.0.smithi138.stdout:707: read oid 16 snap -1 2023-06-27T12:41:22.578 INFO:tasks.rados.rados.0.smithi138.stdout:707: expect deleted 2023-06-27T12:41:22.578 INFO:tasks.rados.rados.0.smithi138.stdout:708: read oid 37 snap 81 2023-06-27T12:41:22.579 INFO:tasks.rados.rados.0.smithi138.stdout:708: expect deleted 2023-06-27T12:41:22.579 INFO:tasks.rados.rados.0.smithi138.stdout:709: read oid 10 snap -1 2023-06-27T12:41:22.579 INFO:tasks.rados.rados.0.smithi138.stdout:709: expect deleted 2023-06-27T12:41:22.579 INFO:tasks.rados.rados.0.smithi138.stdout:710: delete oid 3 current snap is 84 2023-06-27T12:41:22.638 INFO:tasks.rados.rados.0.smithi138.stdout:706: expect (ObjNum 245 snap 68 seq_num 245) 2023-06-27T12:41:22.748 INFO:tasks.rados.rados.0.smithi138.stdout:705: finishing write tid 1 to smithi138138488-13 2023-06-27T12:41:22.748 INFO:tasks.rados.rados.0.smithi138.stdout:705: finishing write tid 2 to smithi138138488-13 2023-06-27T12:41:22.748 INFO:tasks.rados.rados.0.smithi138.stdout:705: finishing write tid 3 to smithi138138488-13 2023-06-27T12:41:22.749 INFO:tasks.rados.rados.0.smithi138.stdout:705: finishing write tid 4 to smithi138138488-13 2023-06-27T12:41:22.749 INFO:tasks.rados.rados.0.smithi138.stdout:705: finishing write tid 5 to smithi138138488-13 2023-06-27T12:41:22.749 INFO:tasks.rados.rados.0.smithi138.stdout:705: oid 13 updating version 0 to 269 2023-06-27T12:41:22.749 INFO:tasks.rados.rados.0.smithi138.stdout:705: oid 13 version 269 is already newer than 269 2023-06-27T12:41:22.749 INFO:tasks.rados.rados.0.smithi138.stdout:705: oid 13 version 269 is already newer than 269 2023-06-27T12:41:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:705: oid 13 version 269 is already newer than 269 2023-06-27T12:41:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 269 (ObjNum 338 snap 84 seq_num 338) dirty exists 2023-06-27T12:41:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:705: left oid 13 (ObjNum 338 snap 84 seq_num 338) 2023-06-27T12:41:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:705: done (5 left) 2023-06-27T12:41:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:706: done (4 left) 2023-06-27T12:41:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:707: done (3 left) 2023-06-27T12:41:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:708: done (2 left) 2023-06-27T12:41:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:709: done (1 left) 2023-06-27T12:41:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:710: done (0 left) 2023-06-27T12:41:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:711: write oid 50 current snap is 84 2023-06-27T12:41:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:711: seq_num 340 ranges {514380=704192,1629760=478435,2604288=442602} 2023-06-27T12:41:22.754 INFO:tasks.rados.rados.0.smithi138.stdout:711: writing smithi138138488-50 from 514380 to 1218572 tid 1 2023-06-27T12:41:22.759 INFO:tasks.rados.rados.0.smithi138.stdout:711: writing smithi138138488-50 from 1629760 to 2108195 tid 2 2023-06-27T12:41:22.764 INFO:tasks.rados.rados.0.smithi138.stdout:711: writing smithi138138488-50 from 2604288 to 3046890 tid 3 2023-06-27T12:41:22.765 INFO:tasks.rados.rados.0.smithi138.stdout:712: snap_remove snap 81 2023-06-27T12:41:23.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:23.501 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:23.496+0000 7f6c0db6e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:23.504 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:23.500+0000 7f6c0db6e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:23.548 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:23.551 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:23.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:23.558 INFO:tasks.rados.rados.0.smithi138.stdout:712: done (1 left) 2023-06-27T12:41:23.558 INFO:tasks.rados.rados.0.smithi138.stdout:713: write (excl) oid 1 current snap is 84 2023-06-27T12:41:23.559 INFO:tasks.rados.rados.0.smithi138.stdout:711: finishing write tid 1 to smithi138138488-50 2023-06-27T12:41:23.559 INFO:tasks.rados.rados.0.smithi138.stdout:713: seq_num 341 ranges {531187=779441,2088242=738446,3431711=299476} 2023-06-27T12:41:23.565 INFO:tasks.rados.rados.0.smithi138.stdout:713: writing smithi138138488-1 from 531187 to 1310628 tid 1 2023-06-27T12:41:23.575 INFO:tasks.rados.rados.0.smithi138.stdout:713: writing smithi138138488-1 from 2088242 to 2826688 tid 2 2023-06-27T12:41:23.582 INFO:tasks.rados.rados.0.smithi138.stdout:713: writing smithi138138488-1 from 3431711 to 3731187 tid 3 2023-06-27T12:41:23.582 INFO:tasks.rados.rados.0.smithi138.stdout:714: write oid 34 current snap is 84 2023-06-27T12:41:23.582 INFO:tasks.rados.rados.0.smithi138.stdout:711: finishing write tid 2 to smithi138138488-50 2023-06-27T12:41:23.582 INFO:tasks.rados.rados.0.smithi138.stdout:714: seq_num 342 ranges {547994=454690,1746724=482104,2659134=465146,3605976=142018} 2023-06-27T12:41:23.589 INFO:tasks.rados.rados.0.smithi138.stdout:714: writing smithi138138488-34 from 547994 to 1002684 tid 1 2023-06-27T12:41:23.598 INFO:tasks.rados.rados.0.smithi138.stdout:714: writing smithi138138488-34 from 1746724 to 2228828 tid 2 2023-06-27T12:41:23.605 INFO:tasks.rados.rados.0.smithi138.stdout:714: writing smithi138138488-34 from 2659134 to 3124280 tid 3 2023-06-27T12:41:23.610 INFO:tasks.rados.rados.0.smithi138.stdout:714: writing smithi138138488-34 from 3605976 to 3747994 tid 4 2023-06-27T12:41:23.610 INFO:tasks.rados.rados.0.smithi138.stdout:715: read oid 9 snap 83 2023-06-27T12:41:23.610 INFO:tasks.rados.rados.0.smithi138.stdout:715: expect (ObjNum 329 snap 83 seq_num 329) 2023-06-27T12:41:23.610 INFO:tasks.rados.rados.0.smithi138.stdout:716: read oid 24 snap -1 2023-06-27T12:41:23.610 INFO:tasks.rados.rados.0.smithi138.stdout:716: expect deleted 2023-06-27T12:41:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:717: snap_create 2023-06-27T12:41:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:711: finishing write tid 3 to smithi138138488-50 2023-06-27T12:41:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:711: finishing write tid 4 to smithi138138488-50 2023-06-27T12:41:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:711: finishing write tid 5 to smithi138138488-50 2023-06-27T12:41:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:711: oid 50 updating version 0 to 56 2023-06-27T12:41:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:711: oid 50 version 56 is already newer than 56 2023-06-27T12:41:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:711: oid 50 version 56 is already newer than 56 2023-06-27T12:41:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:711: oid 50 version 56 is already newer than 56 2023-06-27T12:41:23.612 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 56 (ObjNum 340 snap 84 seq_num 340) dirty exists 2023-06-27T12:41:23.612 INFO:tasks.rados.rados.0.smithi138.stdout:711: left oid 50 (ObjNum 340 snap 84 seq_num 340) 2023-06-27T12:41:23.678 INFO:tasks.rados.rados.0.smithi138.stdout:713: finishing write tid 1 to smithi138138488-1 2023-06-27T12:41:23.685 INFO:tasks.rados.rados.0.smithi138.stdout:714: finishing write tid 1 to smithi138138488-34 2023-06-27T12:41:23.690 INFO:tasks.rados.rados.0.smithi138.stdout:714: finishing write tid 2 to smithi138138488-34 2023-06-27T12:41:23.706 INFO:tasks.rados.rados.0.smithi138.stdout:713: finishing write tid 2 to smithi138138488-1 2023-06-27T12:41:23.707 INFO:tasks.rados.rados.0.smithi138.stdout:714: finishing write tid 3 to smithi138138488-34 2023-06-27T12:41:23.712 INFO:tasks.rados.rados.0.smithi138.stdout:714: finishing write tid 4 to smithi138138488-34 2023-06-27T12:41:23.713 INFO:tasks.rados.rados.0.smithi138.stdout:714: finishing write tid 5 to smithi138138488-34 2023-06-27T12:41:23.715 INFO:tasks.rados.rados.0.smithi138.stdout:714: finishing write tid 6 to smithi138138488-34 2023-06-27T12:41:23.715 INFO:tasks.rados.rados.0.smithi138.stdout:714: oid 34 updating version 0 to 249 2023-06-27T12:41:23.715 INFO:tasks.rados.rados.0.smithi138.stdout:714: oid 34 version 249 is already newer than 249 2023-06-27T12:41:23.715 INFO:tasks.rados.rados.0.smithi138.stdout:714: oid 34 version 249 is already newer than 249 2023-06-27T12:41:23.715 INFO:tasks.rados.rados.0.smithi138.stdout:714: oid 34 version 249 is already newer than 249 2023-06-27T12:41:23.716 INFO:tasks.rados.rados.0.smithi138.stdout:714: oid 34 version 249 is already newer than 249 2023-06-27T12:41:23.716 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 249 (ObjNum 342 snap 84 seq_num 342) dirty exists 2023-06-27T12:41:23.716 INFO:tasks.rados.rados.0.smithi138.stdout:714: left oid 34 (ObjNum 342 snap 84 seq_num 342) 2023-06-27T12:41:23.748 INFO:tasks.rados.rados.0.smithi138.stdout:713: finishing write tid 3 to smithi138138488-1 2023-06-27T12:41:23.748 INFO:tasks.rados.rados.0.smithi138.stdout:713: finishing write tid 4 to smithi138138488-1 2023-06-27T12:41:23.749 INFO:tasks.rados.rados.0.smithi138.stdout:713: finishing write tid 5 to smithi138138488-1 2023-06-27T12:41:23.750 INFO:tasks.rados.rados.0.smithi138.stdout:713: oid 1 updating version 0 to 202 2023-06-27T12:41:23.750 INFO:tasks.rados.rados.0.smithi138.stdout:713: oid 1 version 202 is already newer than 202 2023-06-27T12:41:23.750 INFO:tasks.rados.rados.0.smithi138.stdout:713: oid 1 version 202 is already newer than 202 2023-06-27T12:41:23.750 INFO:tasks.rados.rados.0.smithi138.stdout:713: oid 1 version 202 is already newer than 199 2023-06-27T12:41:23.750 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 202 (ObjNum 341 snap 84 seq_num 341) dirty exists 2023-06-27T12:41:23.750 INFO:tasks.rados.rados.0.smithi138.stdout:713: left oid 1 (ObjNum 341 snap 84 seq_num 341) 2023-06-27T12:41:23.764 INFO:tasks.rados.rados.0.smithi138.stdout:715: expect (ObjNum 329 snap 83 seq_num 329) 2023-06-27T12:41:24.567 INFO:tasks.rados.rados.0.smithi138.stdout:711: done (5 left) 2023-06-27T12:41:24.567 INFO:tasks.rados.rados.0.smithi138.stdout:713: done (4 left) 2023-06-27T12:41:24.567 INFO:tasks.rados.rados.0.smithi138.stdout:714: done (3 left) 2023-06-27T12:41:24.567 INFO:tasks.rados.rados.0.smithi138.stdout:715: done (2 left) 2023-06-27T12:41:24.567 INFO:tasks.rados.rados.0.smithi138.stdout:716: done (1 left) 2023-06-27T12:41:24.568 INFO:tasks.rados.rados.0.smithi138.stdout:717: done (0 left) 2023-06-27T12:41:24.568 INFO:tasks.rados.rados.0.smithi138.stdout:718: delete oid 2 current snap is 85 2023-06-27T12:41:24.644 INFO:tasks.rados.rados.0.smithi138.stdout:718: done (0 left) 2023-06-27T12:41:24.644 INFO:tasks.rados.rados.0.smithi138.stdout:719: read oid 14 snap -1 2023-06-27T12:41:24.644 INFO:tasks.rados.rados.0.smithi138.stdout:719: expect (ObjNum 218 snap 49 seq_num 218) 2023-06-27T12:41:24.644 INFO:tasks.rados.rados.0.smithi138.stdout:720: read oid 50 snap 84 2023-06-27T12:41:24.644 INFO:tasks.rados.rados.0.smithi138.stdout:720: expect (ObjNum 340 snap 84 seq_num 340) 2023-06-27T12:41:24.645 INFO:tasks.rados.rados.0.smithi138.stdout:721: read oid 43 snap 84 2023-06-27T12:41:24.645 INFO:tasks.rados.rados.0.smithi138.stdout:721: expect (ObjNum 222 snap 50 seq_num 222) 2023-06-27T12:41:24.645 INFO:tasks.rados.rados.0.smithi138.stdout:722: snap_remove snap 83 2023-06-27T12:41:24.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~50s 2023-06-27T12:41:25.417 INFO:tasks.rados.rados.0.smithi138.stdout:722: done (3 left) 2023-06-27T12:41:25.418 INFO:tasks.rados.rados.0.smithi138.stdout:723: snap_create 2023-06-27T12:41:25.418 INFO:tasks.rados.rados.0.smithi138.stdout:721: expect (ObjNum 222 snap 50 seq_num 222) 2023-06-27T12:41:25.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:25.561 INFO:tasks.rados.rados.0.smithi138.stdout:719: expect (ObjNum 218 snap 49 seq_num 218) 2023-06-27T12:41:25.684 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:25.679+0000 7f1ae1ad5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:25.687 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:25.683+0000 7f1ae1ad5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:25.732 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:25.736 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:25.737 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:25.834 INFO:tasks.rados.rados.0.smithi138.stdout:720: expect (ObjNum 340 snap 84 seq_num 340) 2023-06-27T12:41:26.080 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:26.081 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:26.081 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:26.209 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:26.202+0000 7fe8eb4ef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:26.212 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:26.208+0000 7fe8eb4ef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:26.402 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:26.404 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:26.416 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-06-27T12:41:25.947455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":71027042,"num_objects":184,"num_object_clones":143,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1032,"num_read_kb":721435,"num_write":867,"num_write_kb":252094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1845,"ondisk_log_size":1845,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":746796,"kb_used_data":623116,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376740564,"statfs":{"total":386547056640,"available":385782337536,"internally_reserved":0,"allocated":638070784,"data_stored":746112174,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6084167,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":35,"num_read_kb":39855,"num_write":21,"num_write_kb":6406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000764"},"pg_stats":[{"pgid":"3.f","version":"180'7","reported_seq":445,"reported_epoch":190,"state":"active+undersized","last_fresh":"2023-06-27T12:41:23.614902+0000","last_change":"2023-06-27T12:41:23.626611+0000","last_active":"2023-06-27T12:41:23.614902+0000","last_peered":"2023-06-27T12:41:23.614902+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:23.614902+0000","last_undegraded":"2023-06-27T12:41:23.614902+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"179'108","reported_seq":598,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.588882+0000","last_change":"2023-06-27T12:41:23.590534+0000","last_active":"2023-06-27T12:41:23.588882+0000","last_peered":"2023-06-27T12:41:23.588882+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:23.588882+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395047,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":85,"num_read_kb":96119,"num_write":73,"num_write_kb":21776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"148'138","reported_seq":575,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.639951+0000","last_change":"2023-06-27T12:41:23.645639+0000","last_active":"2023-06-27T12:41:23.639951+0000","last_peered":"2023-06-27T12:41:23.639951+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:23.639951+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":15,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":31658,"num_write":97,"num_write_kb":28429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"184'74","reported_seq":530,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.660471+0000","last_change":"2023-06-27T12:41:23.664526+0000","last_active":"2023-06-27T12:41:23.660471+0000","last_peered":"2023-06-27T12:41:23.660471+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:23.660471+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":24,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563117,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":66,"num_read_kb":31562,"num_write":56,"num_write_kb":15640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":132,"reported_epoch":190,"state":"active+undersized","last_fresh":"2023-06-27T12:41:23.602379+0000","last_change":"2023-06-27T12:41:23.610722+0000","last_active":"2023-06-27T12:41:23.602379+0000","last_peered":"2023-06-27T12:41:23.602379+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:23.602379+0000","last_undegraded":"2023-06-27T12:41:23.602379+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":24671,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"190'249","reported_seq":587,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-06-27T12:41:23.885462+0000","last_change":"2023-06-27T12:41:23.886806+0000","last_active":"2023-06-27T12:41:23.885462+0000","last_peered":"2023-06-27T12:41:23.885462+0000","last_clean":"2023-06-27T12:41:23.885462+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:23.885462+0000","last_undegraded":"2023-06-27T12:41:23.885462+0000","last_fullsized":"2023-06-27T12:41:23.885462+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6941357,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":179,"num_read_kb":118395,"num_write":133,"num_write_kb":38521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"184'134","reported_seq":483,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-06-27T12:41:23.884050+0000","last_change":"2023-06-27T12:41:23.885589+0000","last_active":"2023-06-27T12:41:23.884050+0000","last_peered":"2023-06-27T12:41:23.884050+0000","last_clean":"2023-06-27T12:41:23.884050+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:23.884050+0000","last_undegraded":"2023-06-27T12:41:23.884050+0000","last_fullsized":"2023-06-27T12:41:23.884050+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3546310,"num_objects":17,"num_object_clones":14,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":75,"num_read_kb":44588,"num_write":72,"num_write_kb":21091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"189'56","reported_seq":510,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.577053+0000","last_change":"2023-06-27T12:41:23.579815+0000","last_active":"2023-06-27T12:41:23.577053+0000","last_peered":"2023-06-27T12:41:23.577053+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:23.577053+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714380,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":34034,"num_write":32,"num_write_kb":10063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":304,"reported_epoch":190,"state":"active+undersized","last_fresh":"2023-06-27T12:41:23.566069+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:23.566069+0000","last_peered":"2023-06-27T12:41:23.566069+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:23.566069+0000","last_undegraded":"2023-06-27T12:41:23.566069+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":440,"reported_epoch":190,"state":"active+undersized","last_fresh":"2023-06-27T12:41:23.576387+0000","last_change":"2023-06-27T12:41:23.600922+0000","last_active":"2023-06-27T12:41:23.576387+0000","last_peered":"2023-06-27T12:41:23.576387+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:23.576387+0000","last_undegraded":"2023-06-27T12:41:23.576387+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":289,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-06-27T12:41:23.611829+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:23.611829+0000","last_peered":"2023-06-27T12:41:23.611829+0000","last_clean":"2023-06-27T12:41:23.611829+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:23.611829+0000","last_undegraded":"2023-06-27T12:41:23.611829+0000","last_fullsized":"2023-06-27T12:41:23.611829+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"188'101","reported_seq":559,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.564850+0000","last_change":"2023-06-27T12:41:23.622820+0000","last_active":"2023-06-27T12:41:23.564850+0000","last_peered":"2023-06-27T12:41:23.564850+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:23.564850+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6319465,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":51,"num_read_kb":51261,"num_write":64,"num_write_kb":17475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":13}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":292,"reported_epoch":190,"state":"active+undersized","last_fresh":"2023-06-27T12:41:23.571617+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:23.571617+0000","last_peered":"2023-06-27T12:41:23.571617+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:23.571617+0000","last_undegraded":"2023-06-27T12:41:23.571617+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"189'269","reported_seq":176,"reported_epoch":190,"state":"active+undersized+degraded+snaptrim","last_fresh":"2023-06-27T12:41:23.567257+0000","last_change":"2023-06-27T12:41:22.578392+0000","last_active":"2023-06-27T12:41:23.567257+0000","last_peered":"2023-06-27T12:41:23.567257+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:23.567257+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":5,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5344692,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":9352,"num_write":29,"num_write_kb":8559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":5}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":289,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-06-27T12:41:23.575901+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:23.575901+0000","last_peered":"2023-06-27T12:41:23.575901+0000","last_clean":"2023-06-27T12:41:23.575901+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:23.575901+0000","last_undegraded":"2023-06-27T12:41:23.575901+0000","last_fullsized":"2023-06-27T12:41:23.575901+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":41,"reported_epoch":190,"state":"active+undersized","last_fresh":"2023-06-27T12:41:23.603440+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:23.603440+0000","last_peered":"2023-06-27T12:41:23.603440+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:23.603440+0000","last_undegraded":"2023-06-27T12:41:23.603440+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"190'202","reported_seq":642,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.757395+0000","last_change":"2023-06-27T12:41:23.601839+0000","last_active":"2023-06-27T12:41:23.757395+0000","last_peered":"2023-06-27T12:41:23.757395+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:23.757395+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":39,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16975202,"num_objects":29,"num_object_clones":23,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":178,"num_read_kb":101235,"num_write":151,"num_write_kb":43525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":29}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":289,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-06-27T12:41:23.598968+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:23.598968+0000","last_peered":"2023-06-27T12:41:23.598968+0000","last_clean":"2023-06-27T12:41:23.598968+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:23.598968+0000","last_undegraded":"2023-06-27T12:41:23.598968+0000","last_fullsized":"2023-06-27T12:41:23.598968+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'87","reported_seq":156,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.573168+0000","last_change":"2023-06-27T12:41:23.592952+0000","last_active":"2023-06-27T12:41:23.573168+0000","last_peered":"2023-06-27T12:41:23.573168+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:23.573168+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":52,"num_read_kb":35679,"num_write":13,"num_write_kb":4215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":190,"state":"active+undersized","last_fresh":"2023-06-27T12:41:23.566275+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:23.566275+0000","last_peered":"2023-06-27T12:41:23.566275+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:23.566275+0000","last_undegraded":"2023-06-27T12:41:23.566275+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"189'109","reported_seq":579,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.574619+0000","last_change":"2023-06-27T12:41:23.575185+0000","last_active":"2023-06-27T12:41:23.574619+0000","last_peered":"2023-06-27T12:41:23.574619+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:23.574619+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9277530,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":67,"num_read_kb":69071,"num_write":77,"num_write_kb":22880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"184'67","reported_seq":149,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.597909+0000","last_change":"2023-06-27T12:41:23.599742+0000","last_active":"2023-06-27T12:41:23.597909+0000","last_peered":"2023-06-27T12:41:23.597909+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:23.597909+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":17010,"num_write":9,"num_write_kb":2650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":304,"reported_epoch":190,"state":"active+undersized","last_fresh":"2023-06-27T12:41:23.568865+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:23.568865+0000","last_peered":"2023-06-27T12:41:23.568865+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:23.568865+0000","last_undegraded":"2023-06-27T12:41:23.568865+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":289,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-06-27T12:41:23.576067+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:23.576067+0000","last_peered":"2023-06-27T12:41:23.576067+0000","last_clean":"2023-06-27T12:41:23.576067+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:23.576067+0000","last_undegraded":"2023-06-27T12:41:23.576067+0000","last_fullsized":"2023-06-27T12:41:23.576067+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":565,"reported_epoch":190,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:23.567887+0000","last_change":"2023-06-27T12:41:23.572373+0000","last_active":"2023-06-27T12:41:23.567887+0000","last_peered":"2023-06-27T12:41:23.567887+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:23.567887+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":60,"num_read_kb":56799,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":71027042,"num_objects":184,"num_object_clones":143,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1032,"num_read_kb":721435,"num_write":867,"num_write_kb":252094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1748,"ondisk_log_size":1748,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":172412,"kb_used_data":141492,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94199428,"statfs":{"total":96636764160,"available":96460214272,"internally_reserved":0,"allocated":144887808,"data_stored":169338566,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":190284,"kb_used_data":159364,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94181556,"statfs":{"total":96636764160,"available":96441913344,"internally_reserved":0,"allocated":163188736,"data_stored":192507942,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":202820,"kb_used_data":171900,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94169020,"statfs":{"total":96636764160,"available":96429076480,"internally_reserved":0,"allocated":176025600,"data_stored":207226114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:26.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.3 3.2 3.4 3.7 2023-06-27T12:41:26.419 INFO:tasks.rados.rados.0.smithi138.stdout:719: done (3 left) 2023-06-27T12:41:26.419 INFO:tasks.rados.rados.0.smithi138.stdout:720: done (2 left) 2023-06-27T12:41:26.419 INFO:tasks.rados.rados.0.smithi138.stdout:721: done (1 left) 2023-06-27T12:41:26.419 INFO:tasks.rados.rados.0.smithi138.stdout:723: done (0 left) 2023-06-27T12:41:26.419 INFO:tasks.rados.rados.0.smithi138.stdout:724: snap_remove snap 84 2023-06-27T12:41:26.550 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:26.539+0000 7f7ecac8f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:26.553 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:26.549+0000 7f7ecac8f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:26.747 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.2,3.7] on osd.1 to force-recovery; instructing pg(s) [3.3,3.4] on osd.2 to force-recovery; 2023-06-27T12:41:26.747 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:41:27.415 INFO:tasks.rados.rados.0.smithi138.stdout:724: done (0 left) 2023-06-27T12:41:27.416 INFO:tasks.rados.rados.0.smithi138.stdout:725: read oid 3 snap -1 2023-06-27T12:41:27.416 INFO:tasks.rados.rados.0.smithi138.stdout:725: expect deleted 2023-06-27T12:41:27.416 INFO:tasks.rados.rados.0.smithi138.stdout:726: write oid 12 current snap is 86 2023-06-27T12:41:27.416 INFO:tasks.rados.rados.0.smithi138.stdout:726: seq_num 344 ranges {581608=721541,1863688=485773,3113980=604043} 2023-06-27T12:41:27.424 INFO:tasks.rados.rados.0.smithi138.stdout:726: writing smithi138138488-12 from 581608 to 1303149 tid 1 2023-06-27T12:41:27.430 INFO:tasks.rados.rados.0.smithi138.stdout:726: writing smithi138138488-12 from 1863688 to 2349461 tid 2 2023-06-27T12:41:27.440 INFO:tasks.rados.rados.0.smithi138.stdout:726: writing smithi138138488-12 from 3113980 to 3718023 tid 3 2023-06-27T12:41:27.440 INFO:tasks.rados.rados.0.smithi138.stdout:727: write oid 43 current snap is 86 2023-06-27T12:41:27.441 INFO:tasks.rados.rados.0.smithi138.stdout:727: seq_num 345 ranges {598415=796790,1922170=745784,3141403=415315} 2023-06-27T12:41:27.451 INFO:tasks.rados.rados.0.smithi138.stdout:727: writing smithi138138488-43 from 598415 to 1395205 tid 1 2023-06-27T12:41:27.460 INFO:tasks.rados.rados.0.smithi138.stdout:727: writing smithi138138488-43 from 1922170 to 2667954 tid 2 2023-06-27T12:41:27.467 INFO:tasks.rados.rados.0.smithi138.stdout:727: writing smithi138138488-43 from 3141403 to 3556718 tid 3 2023-06-27T12:41:27.467 INFO:tasks.rados.rados.0.smithi138.stdout:728: snap_create 2023-06-27T12:41:27.529 INFO:tasks.rados.rados.0.smithi138.stdout:727: finishing write tid 1 to smithi138138488-43 2023-06-27T12:41:27.570 INFO:tasks.rados.rados.0.smithi138.stdout:727: finishing write tid 2 to smithi138138488-43 2023-06-27T12:41:27.579 INFO:tasks.rados.rados.0.smithi138.stdout:727: finishing write tid 3 to smithi138138488-43 2023-06-27T12:41:27.580 INFO:tasks.rados.rados.0.smithi138.stdout:727: finishing write tid 4 to smithi138138488-43 2023-06-27T12:41:27.581 INFO:tasks.rados.rados.0.smithi138.stdout:727: finishing write tid 5 to smithi138138488-43 2023-06-27T12:41:27.581 INFO:tasks.rados.rados.0.smithi138.stdout:727: oid 43 updating version 0 to 116 2023-06-27T12:41:27.582 INFO:tasks.rados.rados.0.smithi138.stdout:727: oid 43 version 116 is already newer than 116 2023-06-27T12:41:27.582 INFO:tasks.rados.rados.0.smithi138.stdout:727: oid 43 version 116 is already newer than 116 2023-06-27T12:41:27.582 INFO:tasks.rados.rados.0.smithi138.stdout:727: oid 43 version 116 is already newer than 116 2023-06-27T12:41:27.582 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 116 (ObjNum 345 snap 86 seq_num 345) dirty exists 2023-06-27T12:41:27.582 INFO:tasks.rados.rados.0.smithi138.stdout:727: left oid 43 (ObjNum 345 snap 86 seq_num 345) 2023-06-27T12:41:27.589 INFO:tasks.rados.rados.0.smithi138.stdout:726: finishing write tid 1 to smithi138138488-12 2023-06-27T12:41:27.590 INFO:tasks.rados.rados.0.smithi138.stdout:726: finishing write tid 2 to smithi138138488-12 2023-06-27T12:41:27.591 INFO:tasks.rados.rados.0.smithi138.stdout:726: finishing write tid 3 to smithi138138488-12 2023-06-27T12:41:27.592 INFO:tasks.rados.rados.0.smithi138.stdout:726: finishing write tid 4 to smithi138138488-12 2023-06-27T12:41:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:726: finishing write tid 5 to smithi138138488-12 2023-06-27T12:41:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:726: oid 12 updating version 0 to 146 2023-06-27T12:41:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:726: oid 12 version 146 is already newer than 146 2023-06-27T12:41:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:726: oid 12 version 146 is already newer than 146 2023-06-27T12:41:27.594 INFO:tasks.rados.rados.0.smithi138.stdout:726: oid 12 version 146 is already newer than 146 2023-06-27T12:41:27.594 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 146 (ObjNum 344 snap 86 seq_num 344) dirty exists 2023-06-27T12:41:27.594 INFO:tasks.rados.rados.0.smithi138.stdout:726: left oid 12 (ObjNum 344 snap 86 seq_num 344) 2023-06-27T12:41:27.738 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:27.864 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:27.858+0000 7efe05460700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:27.866 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:27.863+0000 7efe05460700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:27.913 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:27.915 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:27.916 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:28.420 INFO:tasks.rados.rados.0.smithi138.stdout:725: done (3 left) 2023-06-27T12:41:28.421 INFO:tasks.rados.rados.0.smithi138.stdout:726: done (2 left) 2023-06-27T12:41:28.421 INFO:tasks.rados.rados.0.smithi138.stdout:727: done (1 left) 2023-06-27T12:41:28.421 INFO:tasks.rados.rados.0.smithi138.stdout:728: done (0 left) 2023-06-27T12:41:28.422 INFO:tasks.rados.rados.0.smithi138.stdout:729: write (excl) oid 19 current snap is 87 2023-06-27T12:41:28.422 INFO:tasks.rados.rados.0.smithi138.stdout:729: seq_num 346 ranges {615222=472039,1580652=489442,2768826=626587} 2023-06-27T12:41:28.427 INFO:tasks.rados.rados.0.smithi138.stdout:729: writing smithi138138488-19 from 615222 to 1087261 tid 1 2023-06-27T12:41:28.433 INFO:tasks.rados.rados.0.smithi138.stdout:729: writing smithi138138488-19 from 1580652 to 2070094 tid 2 2023-06-27T12:41:28.443 INFO:tasks.rados.rados.0.smithi138.stdout:729: writing smithi138138488-19 from 2768826 to 3395413 tid 3 2023-06-27T12:41:28.444 INFO:tasks.rados.rados.0.smithi138.stdout:730: write oid 9 current snap is 87 2023-06-27T12:41:28.444 INFO:tasks.rados.rados.0.smithi138.stdout:730: seq_num 347 ranges {632029=547288,1639134=749453,2796249=437859,3653524=178505} 2023-06-27T12:41:28.452 INFO:tasks.rados.rados.0.smithi138.stdout:730: writing smithi138138488-9 from 632029 to 1179317 tid 1 2023-06-27T12:41:28.461 INFO:tasks.rados.rados.0.smithi138.stdout:730: writing smithi138138488-9 from 1639134 to 2388587 tid 2 2023-06-27T12:41:28.467 INFO:tasks.rados.rados.0.smithi138.stdout:730: writing smithi138138488-9 from 2796249 to 3234108 tid 3 2023-06-27T12:41:28.471 INFO:tasks.rados.rados.0.smithi138.stdout:730: writing smithi138138488-9 from 3653524 to 3832029 tid 4 2023-06-27T12:41:28.471 INFO:tasks.rados.rados.0.smithi138.stdout:731: delete oid 5 current snap is 87 2023-06-27T12:41:28.509 INFO:tasks.rados.rados.0.smithi138.stdout:729: finishing write tid 1 to smithi138138488-19 2023-06-27T12:41:28.533 INFO:tasks.rados.rados.0.smithi138.stdout:729: finishing write tid 2 to smithi138138488-19 2023-06-27T12:41:28.534 INFO:tasks.rados.rados.0.smithi138.stdout:731: done (2 left) 2023-06-27T12:41:28.534 INFO:tasks.rados.rados.0.smithi138.stdout:732: snap_remove snap 85 2023-06-27T12:41:29.419 INFO:tasks.rados.rados.0.smithi138.stdout:732: done (2 left) 2023-06-27T12:41:29.420 INFO:tasks.rados.rados.0.smithi138.stdout:733: read oid 6 snap -1 2023-06-27T12:41:29.420 INFO:tasks.rados.rados.0.smithi138.stdout:733: expect (ObjNum 290 snap 78 seq_num 290) 2023-06-27T12:41:29.420 INFO:tasks.rados.rados.0.smithi138.stdout:734: snap_remove snap 86 2023-06-27T12:41:29.918 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:30.047 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:30.042+0000 7f01d9be2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:30.046+0000 7f01d9be2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:30.095 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:30.098 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:30.098 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:30.346 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~55s 2023-06-27T12:41:30.420 INFO:tasks.rados.rados.0.smithi138.stdout:734: done (3 left) 2023-06-27T12:41:30.420 INFO:tasks.rados.rados.0.smithi138.stdout:735: delete oid 2 current snap is 87 2023-06-27T12:41:30.420 INFO:tasks.rados.rados.0.smithi138.stdout:729: finishing write tid 3 to smithi138138488-19 2023-06-27T12:41:30.420 INFO:tasks.rados.rados.0.smithi138.stdout:729: finishing write tid 4 to smithi138138488-19 2023-06-27T12:41:30.421 INFO:tasks.rados.rados.0.smithi138.stdout:729: finishing write tid 5 to smithi138138488-19 2023-06-27T12:41:30.421 INFO:tasks.rados.rados.0.smithi138.stdout:729: oid 19 updating version 0 to 209 2023-06-27T12:41:30.421 INFO:tasks.rados.rados.0.smithi138.stdout:729: oid 19 version 209 is already newer than 205 2023-06-27T12:41:30.421 INFO:tasks.rados.rados.0.smithi138.stdout:729: oid 19 version 209 is already newer than 209 2023-06-27T12:41:30.421 INFO:tasks.rados.rados.0.smithi138.stdout:729: oid 19 version 209 is already newer than 209 2023-06-27T12:41:30.421 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 209 (ObjNum 346 snap 87 seq_num 346) dirty exists 2023-06-27T12:41:30.421 INFO:tasks.rados.rados.0.smithi138.stdout:729: left oid 19 (ObjNum 346 snap 87 seq_num 346) 2023-06-27T12:41:30.422 INFO:tasks.rados.rados.0.smithi138.stdout:730: finishing write tid 1 to smithi138138488-9 2023-06-27T12:41:30.422 INFO:tasks.rados.rados.0.smithi138.stdout:730: finishing write tid 2 to smithi138138488-9 2023-06-27T12:41:30.422 INFO:tasks.rados.rados.0.smithi138.stdout:730: finishing write tid 3 to smithi138138488-9 2023-06-27T12:41:30.422 INFO:tasks.rados.rados.0.smithi138.stdout:730: finishing write tid 4 to smithi138138488-9 2023-06-27T12:41:30.422 INFO:tasks.rados.rados.0.smithi138.stdout:730: finishing write tid 5 to smithi138138488-9 2023-06-27T12:41:30.422 INFO:tasks.rados.rados.0.smithi138.stdout:730: finishing write tid 6 to smithi138138488-9 2023-06-27T12:41:30.422 INFO:tasks.rados.rados.0.smithi138.stdout:730: oid 9 updating version 0 to 215 2023-06-27T12:41:30.422 INFO:tasks.rados.rados.0.smithi138.stdout:730: oid 9 version 215 is already newer than 215 2023-06-27T12:41:30.423 INFO:tasks.rados.rados.0.smithi138.stdout:730: oid 9 version 215 is already newer than 215 2023-06-27T12:41:30.423 INFO:tasks.rados.rados.0.smithi138.stdout:730: oid 9 version 215 is already newer than 215 2023-06-27T12:41:30.423 INFO:tasks.rados.rados.0.smithi138.stdout:730: oid 9 version 215 is already newer than 215 2023-06-27T12:41:30.423 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 215 (ObjNum 347 snap 87 seq_num 347) dirty exists 2023-06-27T12:41:30.423 INFO:tasks.rados.rados.0.smithi138.stdout:730: left oid 9 (ObjNum 347 snap 87 seq_num 347) 2023-06-27T12:41:30.423 INFO:tasks.rados.rados.0.smithi138.stdout:733: expect (ObjNum 290 snap 78 seq_num 290) 2023-06-27T12:41:30.624 INFO:tasks.rados.rados.0.smithi138.stdout:729: done (3 left) 2023-06-27T12:41:30.624 INFO:tasks.rados.rados.0.smithi138.stdout:730: done (2 left) 2023-06-27T12:41:30.625 INFO:tasks.rados.rados.0.smithi138.stdout:733: done (1 left) 2023-06-27T12:41:30.625 INFO:tasks.rados.rados.0.smithi138.stdout:735: done (0 left) 2023-06-27T12:41:30.625 INFO:tasks.rados.rados.0.smithi138.stdout:736: snap_create 2023-06-27T12:41:31.585 INFO:tasks.rados.rados.0.smithi138.stdout:736: done (0 left) 2023-06-27T12:41:31.585 INFO:tasks.rados.rados.0.smithi138.stdout:737: write oid 12 current snap is 88 2023-06-27T12:41:31.585 INFO:tasks.rados.rados.0.smithi138.stdout:737: seq_num 350 ranges {682450=489388,1814580=496780,2878518=788028} 2023-06-27T12:41:31.591 INFO:tasks.rados.rados.0.smithi138.stdout:737: writing smithi138138488-12 from 682450 to 1171838 tid 1 2023-06-27T12:41:31.598 INFO:tasks.rados.rados.0.smithi138.stdout:737: writing smithi138138488-12 from 1814580 to 2311360 tid 2 2023-06-27T12:41:31.606 INFO:tasks.rados.rados.0.smithi138.stdout:737: writing smithi138138488-12 from 2878518 to 3666546 tid 3 2023-06-27T12:41:31.606 INFO:tasks.rados.rados.0.smithi138.stdout:738: snap_remove snap 87 2023-06-27T12:41:31.759 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:31.759 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:31.759 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:31.887 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:31.880+0000 7f10b2ffe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:31.890 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:31.887+0000 7f10b2ffe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:32.085 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:32.086 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:32.098 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-06-27T12:41:31.949000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":72606933,"num_objects":189,"num_object_clones":148,"num_object_copies":567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":1054,"num_read_kb":742302,"num_write":888,"num_write_kb":259199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1881,"ondisk_log_size":1881,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":741136,"kb_used_data":617456,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376746224,"statfs":{"total":386547056640,"available":385788133376,"internally_reserved":0,"allocated":632274944,"data_stored":739662606,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1579891,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":22,"num_read_kb":20867,"num_write":21,"num_write_kb":7105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001000"},"pg_stats":[{"pgid":"3.f","version":"180'7","reported_seq":455,"reported_epoch":195,"state":"active+undersized","last_fresh":"2023-06-27T12:41:28.446792+0000","last_change":"2023-06-27T12:41:27.475621+0000","last_active":"2023-06-27T12:41:28.446792+0000","last_peered":"2023-06-27T12:41:28.446792+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:28.446792+0000","last_undegraded":"2023-06-27T12:41:28.446792+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"179'108","reported_seq":611,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.825999+0000","last_change":"2023-06-27T12:41:27.437712+0000","last_active":"2023-06-27T12:41:28.825999+0000","last_peered":"2023-06-27T12:41:28.825999+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:28.825999+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395047,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":85,"num_read_kb":96119,"num_write":73,"num_write_kb":21776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"194'146","reported_seq":605,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.460175+0000","last_change":"2023-06-27T12:41:27.487780+0000","last_active":"2023-06-27T12:41:28.460175+0000","last_peered":"2023-06-27T12:41:28.460175+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:28.460175+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10991910,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":77,"num_read_kb":31659,"num_write":102,"num_write_kb":30199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"184'74","reported_seq":551,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.462747+0000","last_change":"2023-06-27T12:41:28.463676+0000","last_active":"2023-06-27T12:41:28.462747+0000","last_peered":"2023-06-27T12:41:28.462747+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:28.462747+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":17,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563117,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":66,"num_read_kb":31562,"num_write":56,"num_write_kb":15640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":142,"reported_epoch":195,"state":"active+undersized","last_fresh":"2023-06-27T12:41:28.432108+0000","last_change":"2023-06-27T12:41:27.473931+0000","last_active":"2023-06-27T12:41:28.432108+0000","last_peered":"2023-06-27T12:41:28.432108+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:28.432108+0000","last_undegraded":"2023-06-27T12:41:28.432108+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":24671,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"190'249","reported_seq":598,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-06-27T12:41:28.443278+0000","last_change":"2023-06-27T12:41:27.461361+0000","last_active":"2023-06-27T12:41:28.443278+0000","last_peered":"2023-06-27T12:41:28.443278+0000","last_clean":"2023-06-27T12:41:28.443278+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:28.443278+0000","last_undegraded":"2023-06-27T12:41:28.443278+0000","last_fullsized":"2023-06-27T12:41:28.443278+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6941357,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":179,"num_read_kb":118395,"num_write":133,"num_write_kb":38521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"184'134","reported_seq":493,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-06-27T12:41:28.438697+0000","last_change":"2023-06-27T12:41:27.475946+0000","last_active":"2023-06-27T12:41:28.438697+0000","last_peered":"2023-06-27T12:41:28.438697+0000","last_clean":"2023-06-27T12:41:28.438697+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:28.438697+0000","last_undegraded":"2023-06-27T12:41:28.438697+0000","last_fullsized":"2023-06-27T12:41:28.438697+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3546310,"num_objects":17,"num_object_clones":14,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":75,"num_read_kb":44588,"num_write":72,"num_write_kb":21091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"189'56","reported_seq":523,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.443715+0000","last_change":"2023-06-27T12:41:27.474418+0000","last_active":"2023-06-27T12:41:28.443715+0000","last_peered":"2023-06-27T12:41:28.443715+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:28.443715+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714380,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":42878,"num_write":32,"num_write_kb":10063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":314,"reported_epoch":195,"state":"active+undersized","last_fresh":"2023-06-27T12:41:28.830214+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:28.830214+0000","last_peered":"2023-06-27T12:41:28.830214+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:28.830214+0000","last_undegraded":"2023-06-27T12:41:28.830214+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":450,"reported_epoch":195,"state":"active+undersized","last_fresh":"2023-06-27T12:41:28.442726+0000","last_change":"2023-06-27T12:41:27.485927+0000","last_active":"2023-06-27T12:41:28.442726+0000","last_peered":"2023-06-27T12:41:28.442726+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:28.442726+0000","last_undegraded":"2023-06-27T12:41:28.442726+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":297,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-06-27T12:41:28.449213+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:28.449213+0000","last_peered":"2023-06-27T12:41:28.449213+0000","last_clean":"2023-06-27T12:41:28.449213+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:28.449213+0000","last_undegraded":"2023-06-27T12:41:28.449213+0000","last_fullsized":"2023-06-27T12:41:28.449213+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"188'101","reported_seq":571,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.829221+0000","last_change":"2023-06-27T12:41:27.436497+0000","last_active":"2023-06-27T12:41:28.829221+0000","last_peered":"2023-06-27T12:41:28.829221+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:28.829221+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6319465,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":51,"num_read_kb":51261,"num_write":64,"num_write_kb":17475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":13}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":300,"reported_epoch":195,"state":"active+undersized","last_fresh":"2023-06-27T12:41:28.438481+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:28.438481+0000","last_peered":"2023-06-27T12:41:28.438481+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:28.438481+0000","last_undegraded":"2023-06-27T12:41:28.438481+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"191'277","reported_seq":196,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.832477+0000","last_change":"2023-06-27T12:41:27.521579+0000","last_active":"2023-06-27T12:41:28.832477+0000","last_peered":"2023-06-27T12:41:28.832477+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:28.832477+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2554697,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":17,"num_read_kb":9352,"num_write":29,"num_write_kb":8559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":6}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":297,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-06-27T12:41:28.443029+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:28.443029+0000","last_peered":"2023-06-27T12:41:28.443029+0000","last_clean":"2023-06-27T12:41:28.443029+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:28.443029+0000","last_undegraded":"2023-06-27T12:41:28.443029+0000","last_fullsized":"2023-06-27T12:41:28.443029+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":49,"reported_epoch":195,"state":"active+undersized","last_fresh":"2023-06-27T12:41:28.431650+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:28.431650+0000","last_peered":"2023-06-27T12:41:28.431650+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:28.431650+0000","last_undegraded":"2023-06-27T12:41:28.431650+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"195'215","reported_seq":683,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.633944+0000","last_change":"2023-06-27T12:41:28.635236+0000","last_active":"2023-06-27T12:41:28.633944+0000","last_peered":"2023-06-27T12:41:28.633944+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:28.633944+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":26,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17496219,"num_objects":31,"num_object_clones":25,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":188,"num_read_kb":110398,"num_write":162,"num_write_kb":46946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":31}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":297,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-06-27T12:41:28.430958+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:28.430958+0000","last_peered":"2023-06-27T12:41:28.430958+0000","last_clean":"2023-06-27T12:41:28.430958+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:28.430958+0000","last_undegraded":"2023-06-27T12:41:28.430958+0000","last_fullsized":"2023-06-27T12:41:28.430958+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'87","reported_seq":167,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.450783+0000","last_change":"2023-06-27T12:41:27.480855+0000","last_active":"2023-06-27T12:41:28.450783+0000","last_peered":"2023-06-27T12:41:28.450783+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:28.450783+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":52,"num_read_kb":35679,"num_write":13,"num_write_kb":4215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":195,"state":"active+undersized","last_fresh":"2023-06-27T12:41:28.828820+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:28.828820+0000","last_peered":"2023-06-27T12:41:28.828820+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:28.828820+0000","last_undegraded":"2023-06-27T12:41:28.828820+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"194'116","reported_seq":598,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.834803+0000","last_change":"2023-06-27T12:41:27.432500+0000","last_active":"2023-06-27T12:41:28.834803+0000","last_peered":"2023-06-27T12:41:28.834803+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:28.834803+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9344791,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":70,"num_read_kb":71930,"num_write":82,"num_write_kb":24794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":16}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"184'67","reported_seq":159,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.444312+0000","last_change":"2023-06-27T12:41:27.474963+0000","last_active":"2023-06-27T12:41:28.444312+0000","last_peered":"2023-06-27T12:41:28.444312+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:28.444312+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":17010,"num_write":9,"num_write_kb":2650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":314,"reported_epoch":195,"state":"active+undersized","last_fresh":"2023-06-27T12:41:28.832480+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:28.832480+0000","last_peered":"2023-06-27T12:41:28.832480+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:28.832480+0000","last_undegraded":"2023-06-27T12:41:28.832480+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":297,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-06-27T12:41:28.439902+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:28.439902+0000","last_peered":"2023-06-27T12:41:28.439902+0000","last_clean":"2023-06-27T12:41:28.439902+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:28.439902+0000","last_undegraded":"2023-06-27T12:41:28.439902+0000","last_fullsized":"2023-06-27T12:41:28.439902+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":577,"reported_epoch":195,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:28.834491+0000","last_change":"2023-06-27T12:41:27.432305+0000","last_active":"2023-06-27T12:41:28.834491+0000","last_peered":"2023-06-27T12:41:28.834491+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:28.834491+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":60,"num_read_kb":56799,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":72606933,"num_objects":189,"num_object_clones":148,"num_object_copies":567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":1054,"num_read_kb":742302,"num_write":888,"num_write_kb":259199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1784,"ondisk_log_size":1784,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":168148,"kb_used_data":137228,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94203692,"statfs":{"total":96636764160,"available":96464580608,"internally_reserved":0,"allocated":140521472,"data_stored":163913184,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":189916,"kb_used_data":158996,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94181924,"statfs":{"total":96636764160,"available":96442290176,"internally_reserved":0,"allocated":162811904,"data_stored":192870582,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":201792,"kb_used_data":170872,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94170048,"statfs":{"total":96636764160,"available":96430129152,"internally_reserved":0,"allocated":174972928,"data_stored":205839288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:32.100 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.c 3.3 3.4 2023-06-27T12:41:32.101 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:32.229 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:32.224+0000 7f57b4854700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:32.232 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:32.228+0000 7f57b4854700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:32.224+0000 7fbaf174d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:32.235+0000 7fbaf174d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:32.290 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:32.293 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:32.294 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:32.441 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c] on osd.0 to force-recovery; instructing pg(s) [3.3,3.4] on osd.2 to force-recovery; 2023-06-27T12:41:32.441 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:41:32.604 INFO:tasks.rados.rados.0.smithi138.stdout:738: done (1 left) 2023-06-27T12:41:32.605 INFO:tasks.rados.rados.0.smithi138.stdout:739: delete oid 31 current snap is 88 2023-06-27T12:41:32.605 INFO:tasks.rados.rados.0.smithi138.stdout:737: finishing write tid 1 to smithi138138488-12 2023-06-27T12:41:32.605 INFO:tasks.rados.rados.0.smithi138.stdout:737: finishing write tid 2 to smithi138138488-12 2023-06-27T12:41:32.605 INFO:tasks.rados.rados.0.smithi138.stdout:737: finishing write tid 3 to smithi138138488-12 2023-06-27T12:41:32.605 INFO:tasks.rados.rados.0.smithi138.stdout:737: finishing write tid 4 to smithi138138488-12 2023-06-27T12:41:32.605 INFO:tasks.rados.rados.0.smithi138.stdout:737: finishing write tid 5 to smithi138138488-12 2023-06-27T12:41:32.606 INFO:tasks.rados.rados.0.smithi138.stdout:737: oid 12 updating version 0 to 151 2023-06-27T12:41:32.606 INFO:tasks.rados.rados.0.smithi138.stdout:737: oid 12 version 151 is already newer than 151 2023-06-27T12:41:32.606 INFO:tasks.rados.rados.0.smithi138.stdout:737: oid 12 version 151 is already newer than 151 2023-06-27T12:41:32.606 INFO:tasks.rados.rados.0.smithi138.stdout:737: oid 12 version 151 is already newer than 150 2023-06-27T12:41:32.606 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 151 (ObjNum 350 snap 88 seq_num 350) dirty exists 2023-06-27T12:41:32.606 INFO:tasks.rados.rados.0.smithi138.stdout:737: left oid 12 (ObjNum 350 snap 88 seq_num 350) 2023-06-27T12:41:32.627 INFO:tasks.rados.rados.0.smithi138.stdout:737: done (1 left) 2023-06-27T12:41:32.627 INFO:tasks.rados.rados.0.smithi138.stdout:739: done (0 left) 2023-06-27T12:41:32.627 INFO:tasks.rados.rados.0.smithi138.stdout:740: write (excl) oid 50 current snap is 88 2023-06-27T12:41:32.628 INFO:tasks.rados.rados.0.smithi138.stdout:740: seq_num 352 ranges {716064=639886,1815191=500449,2817011=526925} 2023-06-27T12:41:32.635 INFO:tasks.rados.rados.0.smithi138.stdout:740: writing smithi138138488-50 from 716064 to 1355950 tid 1 2023-06-27T12:41:32.641 INFO:tasks.rados.rados.0.smithi138.stdout:740: writing smithi138138488-50 from 1815191 to 2315640 tid 2 2023-06-27T12:41:32.649 INFO:tasks.rados.rados.0.smithi138.stdout:740: writing smithi138138488-50 from 2817011 to 3343936 tid 3 2023-06-27T12:41:32.650 INFO:tasks.rados.rados.0.smithi138.stdout:741: snap_create 2023-06-27T12:41:32.706 INFO:tasks.rados.rados.0.smithi138.stdout:740: finishing write tid 1 to smithi138138488-50 2023-06-27T12:41:32.761 INFO:tasks.rados.rados.0.smithi138.stdout:740: finishing write tid 2 to smithi138138488-50 2023-06-27T12:41:32.761 INFO:tasks.rados.rados.0.smithi138.stdout:740: finishing write tid 3 to smithi138138488-50 2023-06-27T12:41:32.762 INFO:tasks.rados.rados.0.smithi138.stdout:740: finishing write tid 4 to smithi138138488-50 2023-06-27T12:41:32.762 INFO:tasks.rados.rados.0.smithi138.stdout:740: finishing write tid 5 to smithi138138488-50 2023-06-27T12:41:32.762 INFO:tasks.rados.rados.0.smithi138.stdout:740: oid 50 updating version 0 to 60 2023-06-27T12:41:32.762 INFO:tasks.rados.rados.0.smithi138.stdout:740: oid 50 version 60 is already newer than 60 2023-06-27T12:41:32.763 INFO:tasks.rados.rados.0.smithi138.stdout:740: oid 50 version 60 is already newer than 57 2023-06-27T12:41:32.763 INFO:tasks.rados.rados.0.smithi138.stdout:740: oid 50 version 60 is already newer than 60 2023-06-27T12:41:32.763 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 60 (ObjNum 352 snap 88 seq_num 352) dirty exists 2023-06-27T12:41:32.763 INFO:tasks.rados.rados.0.smithi138.stdout:740: left oid 50 (ObjNum 352 snap 88 seq_num 352) 2023-06-27T12:41:33.615 INFO:tasks.rados.rados.0.smithi138.stdout:740: done (1 left) 2023-06-27T12:41:33.615 INFO:tasks.rados.rados.0.smithi138.stdout:741: done (0 left) 2023-06-27T12:41:33.615 INFO:tasks.rados.rados.0.smithi138.stdout:742: read oid 17 snap -1 2023-06-27T12:41:33.616 INFO:tasks.rados.rados.0.smithi138.stdout:742: expect (ObjNum 321 snap 81 seq_num 321) 2023-06-27T12:41:33.616 INFO:tasks.rados.rados.0.smithi138.stdout:743: snap_create 2023-06-27T12:41:33.654 INFO:tasks.rados.rados.0.smithi138.stdout:742: expect (ObjNum 321 snap 81 seq_num 321) 2023-06-27T12:41:34.295 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:34.423 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:34.418+0000 7fa642383700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:34.425 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:34.422+0000 7fa642383700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:34.471 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:34.474 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:34.474 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:34.622 INFO:tasks.rados.rados.0.smithi138.stdout:742: done (1 left) 2023-06-27T12:41:34.623 INFO:tasks.rados.rados.0.smithi138.stdout:743: done (0 left) 2023-06-27T12:41:34.623 INFO:tasks.rados.rados.0.smithi138.stdout:744: read oid 29 snap -1 2023-06-27T12:41:34.623 INFO:tasks.rados.rados.0.smithi138.stdout:744: expect deleted 2023-06-27T12:41:34.623 INFO:tasks.rados.rados.0.smithi138.stdout:745: write oid 35 current snap is 90 2023-06-27T12:41:34.623 INFO:tasks.rados.rados.0.smithi138.stdout:745: seq_num 353 ranges {732871=715135,1873673=644107,3244434=688437} 2023-06-27T12:41:34.632 INFO:tasks.rados.rados.0.smithi138.stdout:745: writing smithi138138488-35 from 732871 to 1448006 tid 1 2023-06-27T12:41:34.640 INFO:tasks.rados.rados.0.smithi138.stdout:745: writing smithi138138488-35 from 1873673 to 2517780 tid 2 2023-06-27T12:41:34.651 INFO:tasks.rados.rados.0.smithi138.stdout:745: writing smithi138138488-35 from 3244434 to 3932871 tid 3 2023-06-27T12:41:34.652 INFO:tasks.rados.rados.0.smithi138.stdout:746: read oid 46 snap -1 2023-06-27T12:41:34.652 INFO:tasks.rados.rados.0.smithi138.stdout:746: expect deleted 2023-06-27T12:41:34.652 INFO:tasks.rados.rados.0.smithi138.stdout:747: read oid 9 snap -1 2023-06-27T12:41:34.652 INFO:tasks.rados.rados.0.smithi138.stdout:747: expect (ObjNum 347 snap 87 seq_num 347) 2023-06-27T12:41:34.652 INFO:tasks.rados.rados.0.smithi138.stdout:748: delete oid 30 current snap is 90 2023-06-27T12:41:34.684 INFO:tasks.rados.rados.0.smithi138.stdout:747: expect (ObjNum 347 snap 87 seq_num 347) 2023-06-27T12:41:35.030 INFO:tasks.rados.rados.0.smithi138.stdout:745: finishing write tid 1 to smithi138138488-35 2023-06-27T12:41:35.030 INFO:tasks.rados.rados.0.smithi138.stdout:745: finishing write tid 2 to smithi138138488-35 2023-06-27T12:41:35.030 INFO:tasks.rados.rados.0.smithi138.stdout:745: finishing write tid 3 to smithi138138488-35 2023-06-27T12:41:35.031 INFO:tasks.rados.rados.0.smithi138.stdout:745: finishing write tid 4 to smithi138138488-35 2023-06-27T12:41:35.031 INFO:tasks.rados.rados.0.smithi138.stdout:745: finishing write tid 5 to smithi138138488-35 2023-06-27T12:41:35.031 INFO:tasks.rados.rados.0.smithi138.stdout:745: oid 35 updating version 0 to 290 2023-06-27T12:41:35.032 INFO:tasks.rados.rados.0.smithi138.stdout:745: oid 35 version 290 is already newer than 290 2023-06-27T12:41:35.032 INFO:tasks.rados.rados.0.smithi138.stdout:745: oid 35 version 290 is already newer than 290 2023-06-27T12:41:35.032 INFO:tasks.rados.rados.0.smithi138.stdout:745: oid 35 version 290 is already newer than 290 2023-06-27T12:41:35.033 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 290 (ObjNum 353 snap 90 seq_num 353) dirty exists 2023-06-27T12:41:35.033 INFO:tasks.rados.rados.0.smithi138.stdout:745: left oid 35 (ObjNum 353 snap 90 seq_num 353) 2023-06-27T12:41:35.033 INFO:tasks.rados.rados.0.smithi138.stdout:744: done (4 left) 2023-06-27T12:41:35.033 INFO:tasks.rados.rados.0.smithi138.stdout:745: done (3 left) 2023-06-27T12:41:35.034 INFO:tasks.rados.rados.0.smithi138.stdout:746: done (2 left) 2023-06-27T12:41:35.034 INFO:tasks.rados.rados.0.smithi138.stdout:747: done (1 left) 2023-06-27T12:41:35.034 INFO:tasks.rados.rados.0.smithi138.stdout:748: done (0 left) 2023-06-27T12:41:35.034 INFO:tasks.rados.rados.0.smithi138.stdout:749: write oid 15 current snap is 90 2023-06-27T12:41:35.035 INFO:tasks.rados.rados.0.smithi138.stdout:749: seq_num 355 ranges {766485=465633,1990637=647776,3299280=667205} 2023-06-27T12:41:35.036 INFO:tasks.rados.rados.0.smithi138.stdout:749: writing smithi138138488-15 from 766485 to 1232118 tid 1 2023-06-27T12:41:35.043 INFO:tasks.rados.rados.0.smithi138.stdout:749: writing smithi138138488-15 from 1990637 to 2638413 tid 2 2023-06-27T12:41:35.051 INFO:tasks.rados.rados.0.smithi138.stdout:749: writing smithi138138488-15 from 3299280 to 3966485 tid 3 2023-06-27T12:41:35.052 INFO:tasks.rados.rados.0.smithi138.stdout:750: read oid 5 snap 89 2023-06-27T12:41:35.052 INFO:tasks.rados.rados.0.smithi138.stdout:750: expect deleted 2023-06-27T12:41:35.052 INFO:tasks.rados.rados.0.smithi138.stdout:751: write oid 25 current snap is 90 2023-06-27T12:41:35.052 INFO:tasks.rados.rados.0.smithi138.stdout:751: seq_num 356 ranges {783292=540882,1932766=507787,3210350=572013} 2023-06-27T12:41:35.058 INFO:tasks.rados.rados.0.smithi138.stdout:751: writing smithi138138488-25 from 783292 to 1324174 tid 1 2023-06-27T12:41:35.066 INFO:tasks.rados.rados.0.smithi138.stdout:751: writing smithi138138488-25 from 1932766 to 2440553 tid 2 2023-06-27T12:41:35.074 INFO:tasks.rados.rados.0.smithi138.stdout:751: writing smithi138138488-25 from 3210350 to 3782363 tid 3 2023-06-27T12:41:35.074 INFO:tasks.rados.rados.0.smithi138.stdout:752: write (excl) oid 44 current snap is 90 2023-06-27T12:41:35.074 INFO:tasks.rados.rados.0.smithi138.stdout:752: seq_num 357 ranges {400099=616131,1591248=408851} 2023-06-27T12:41:35.081 INFO:tasks.rados.rados.0.smithi138.stdout:752: writing smithi138138488-44 from 400099 to 1016230 tid 1 2023-06-27T12:41:35.089 INFO:tasks.rados.rados.0.smithi138.stdout:752: writing smithi138138488-44 from 1591248 to 2000099 tid 2 2023-06-27T12:41:35.089 INFO:tasks.rados.rados.0.smithi138.stdout:753: delete oid 22 current snap is 90 2023-06-27T12:41:35.090 INFO:tasks.rados.rados.0.smithi138.stdout:749: finishing write tid 1 to smithi138138488-15 2023-06-27T12:41:35.090 INFO:tasks.rados.rados.0.smithi138.stdout:749: finishing write tid 2 to smithi138138488-15 2023-06-27T12:41:35.090 INFO:tasks.rados.rados.0.smithi138.stdout:749: finishing write tid 3 to smithi138138488-15 2023-06-27T12:41:35.094 INFO:tasks.rados.rados.0.smithi138.stdout:751: finishing write tid 1 to smithi138138488-25 2023-06-27T12:41:35.113 INFO:tasks.rados.rados.0.smithi138.stdout:749: finishing write tid 4 to smithi138138488-15 2023-06-27T12:41:35.114 INFO:tasks.rados.rados.0.smithi138.stdout:749: finishing write tid 5 to smithi138138488-15 2023-06-27T12:41:35.114 INFO:tasks.rados.rados.0.smithi138.stdout:749: oid 15 updating version 0 to 37 2023-06-27T12:41:35.114 INFO:tasks.rados.rados.0.smithi138.stdout:749: oid 15 version 37 is already newer than 37 2023-06-27T12:41:35.114 INFO:tasks.rados.rados.0.smithi138.stdout:749: oid 15 version 37 is already newer than 37 2023-06-27T12:41:35.114 INFO:tasks.rados.rados.0.smithi138.stdout:749: oid 15 version 37 is already newer than 35 2023-06-27T12:41:35.114 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 37 (ObjNum 355 snap 90 seq_num 355) dirty exists 2023-06-27T12:41:35.115 INFO:tasks.rados.rados.0.smithi138.stdout:749: left oid 15 (ObjNum 355 snap 90 seq_num 355) 2023-06-27T12:41:35.135 INFO:tasks.rados.rados.0.smithi138.stdout:751: finishing write tid 2 to smithi138138488-25 2023-06-27T12:41:35.135 INFO:tasks.rados.rados.0.smithi138.stdout:751: finishing write tid 3 to smithi138138488-25 2023-06-27T12:41:35.135 INFO:tasks.rados.rados.0.smithi138.stdout:751: finishing write tid 4 to smithi138138488-25 2023-06-27T12:41:35.135 INFO:tasks.rados.rados.0.smithi138.stdout:751: finishing write tid 5 to smithi138138488-25 2023-06-27T12:41:35.135 INFO:tasks.rados.rados.0.smithi138.stdout:751: oid 25 updating version 0 to 77 2023-06-27T12:41:35.136 INFO:tasks.rados.rados.0.smithi138.stdout:751: oid 25 version 77 is already newer than 77 2023-06-27T12:41:35.136 INFO:tasks.rados.rados.0.smithi138.stdout:751: oid 25 version 77 is already newer than 77 2023-06-27T12:41:35.136 INFO:tasks.rados.rados.0.smithi138.stdout:751: oid 25 version 77 is already newer than 77 2023-06-27T12:41:35.136 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 77 (ObjNum 356 snap 90 seq_num 356) dirty exists 2023-06-27T12:41:35.136 INFO:tasks.rados.rados.0.smithi138.stdout:751: left oid 25 (ObjNum 356 snap 90 seq_num 356) 2023-06-27T12:41:35.137 INFO:tasks.rados.rados.0.smithi138.stdout:749: done (4 left) 2023-06-27T12:41:35.137 INFO:tasks.rados.rados.0.smithi138.stdout:750: done (3 left) 2023-06-27T12:41:35.137 INFO:tasks.rados.rados.0.smithi138.stdout:751: done (2 left) 2023-06-27T12:41:35.137 INFO:tasks.rados.rados.0.smithi138.stdout:753: done (1 left) 2023-06-27T12:41:35.138 INFO:tasks.rados.rados.0.smithi138.stdout:754: delete oid 29 current snap is 90 2023-06-27T12:41:35.138 INFO:tasks.rados.rados.0.smithi138.stdout:754: done (1 left) 2023-06-27T12:41:35.138 INFO:tasks.rados.rados.0.smithi138.stdout:755: snap_create 2023-06-27T12:41:35.154 INFO:tasks.rados.rados.0.smithi138.stdout:752: finishing write tid 1 to smithi138138488-44 2023-06-27T12:41:35.199 INFO:tasks.rados.rados.0.smithi138.stdout:752: finishing write tid 2 to smithi138138488-44 2023-06-27T12:41:35.200 INFO:tasks.rados.rados.0.smithi138.stdout:752: finishing write tid 3 to smithi138138488-44 2023-06-27T12:41:35.200 INFO:tasks.rados.rados.0.smithi138.stdout:752: finishing write tid 4 to smithi138138488-44 2023-06-27T12:41:35.201 INFO:tasks.rados.rados.0.smithi138.stdout:752: oid 44 updating version 0 to 256 2023-06-27T12:41:35.201 INFO:tasks.rados.rados.0.smithi138.stdout:752: oid 44 version 256 is already newer than 256 2023-06-27T12:41:35.201 INFO:tasks.rados.rados.0.smithi138.stdout:752: oid 44 version 256 is already newer than 253 2023-06-27T12:41:35.201 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 256 (ObjNum 357 snap 90 seq_num 357) dirty exists 2023-06-27T12:41:35.201 INFO:tasks.rados.rados.0.smithi138.stdout:752: left oid 44 (ObjNum 357 snap 90 seq_num 357) 2023-06-27T12:41:35.417 INFO:tasks.rados.rados.0.smithi138.stdout:752: done (1 left) 2023-06-27T12:41:35.418 INFO:tasks.rados.rados.0.smithi138.stdout:755: done (0 left) 2023-06-27T12:41:35.418 INFO:tasks.rados.rados.0.smithi138.stdout:756: snap_remove snap 88 2023-06-27T12:41:35.852 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~61s 2023-06-27T12:41:36.418 INFO:tasks.rados.rados.0.smithi138.stdout:756: done (0 left) 2023-06-27T12:41:36.418 INFO:tasks.rados.rados.0.smithi138.stdout:757: snap_remove snap 90 2023-06-27T12:41:36.476 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:36.610 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:36.604+0000 7f2b4339f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:36.612 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:36.609+0000 7f2b4339f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:36.658 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:36.661 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:36.662 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:37.419 INFO:tasks.rados.rados.0.smithi138.stdout:757: done (0 left) 2023-06-27T12:41:37.419 INFO:tasks.rados.rados.0.smithi138.stdout:758: write oid 26 current snap is 91 2023-06-27T12:41:37.419 INFO:tasks.rados.rados.0.smithi138.stdout:758: seq_num 360 ranges {450520=558231,1483047=515125} 2023-06-27T12:41:37.424 INFO:tasks.rados.rados.0.smithi138.stdout:758: writing smithi138138488-26 from 450520 to 1008751 tid 1 2023-06-27T12:41:37.430 INFO:tasks.rados.rados.0.smithi138.stdout:758: writing smithi138138488-26 from 1483047 to 1998172 tid 2 2023-06-27T12:41:37.430 INFO:tasks.rados.rados.0.smithi138.stdout:759: read oid 29 snap -1 2023-06-27T12:41:37.430 INFO:tasks.rados.rados.0.smithi138.stdout:759: expect deleted 2023-06-27T12:41:37.431 INFO:tasks.rados.rados.0.smithi138.stdout:760: write (excl) oid 9 current snap is 91 2023-06-27T12:41:37.431 INFO:tasks.rados.rados.0.smithi138.stdout:760: seq_num 361 ranges {467327=633480,1825176=242151} 2023-06-27T12:41:37.436 INFO:tasks.rados.rados.0.smithi138.stdout:760: writing smithi138138488-9 from 467327 to 1100807 tid 1 2023-06-27T12:41:37.443 INFO:tasks.rados.rados.0.smithi138.stdout:760: writing smithi138138488-9 from 1825176 to 2067327 tid 2 2023-06-27T12:41:37.443 INFO:tasks.rados.rados.0.smithi138.stdout:761: snap_remove snap 89 2023-06-27T12:41:37.452 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:37.453 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:37.453 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:37.589+0000 7fc3b5161700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:37.599 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:37.595+0000 7fc3b5161700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:37.795 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:37.806 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-06-27T12:41:37.414158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":72909459,"num_objects":190,"num_object_clones":149,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":1061,"num_read_kb":754550,"num_write":898,"num_write_kb":262562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1923,"ondisk_log_size":1923,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":737748,"kb_used_data":613068,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376749612,"statfs":{"total":386547056640,"available":385791602688,"internally_reserved":0,"allocated":627781632,"data_stored":730600981,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":302526,"num_objects":1,"num_object_clones":1,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12248,"num_write":10,"num_write_kb":3363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.803896"},"pg_stats":[{"pgid":"3.f","version":"180'7","reported_seq":466,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-06-27T12:41:33.808108+0000","last_change":"2023-06-27T12:41:32.619477+0000","last_active":"2023-06-27T12:41:33.808108+0000","last_peered":"2023-06-27T12:41:33.808108+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:33.808108+0000","last_undegraded":"2023-06-27T12:41:33.808108+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"179'108","reported_seq":626,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.641959+0000","last_change":"2023-06-27T12:41:32.622261+0000","last_active":"2023-06-27T12:41:33.641959+0000","last_peered":"2023-06-27T12:41:33.641959+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:33.641959+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395047,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":87,"num_read_kb":102751,"num_write":73,"num_write_kb":21776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"198'151","reported_seq":621,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.624213+0000","last_change":"2023-06-27T12:41:32.711501+0000","last_active":"2023-06-27T12:41:33.624213+0000","last_peered":"2023-06-27T12:41:33.624213+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:33.624213+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11092752,"num_objects":19,"num_object_clones":14,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":78,"num_read_kb":31660,"num_write":107,"num_write_kb":31933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"184'74","reported_seq":569,"reported_epoch":200,"state":"active+undersized+degraded+snaptrim","last_fresh":"2023-06-27T12:41:33.810735+0000","last_change":"2023-06-27T12:41:33.810314+0000","last_active":"2023-06-27T12:41:33.810735+0000","last_peered":"2023-06-27T12:41:33.810735+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:33.810735+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563117,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":66,"num_read_kb":31562,"num_write":56,"num_write_kb":15640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'27","reported_seq":153,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-06-27T12:41:33.807649+0000","last_change":"2023-06-27T12:41:32.621606+0000","last_active":"2023-06-27T12:41:33.807649+0000","last_peered":"2023-06-27T12:41:33.807649+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:33.807649+0000","last_undegraded":"2023-06-27T12:41:33.807649+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":24671,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"190'249","reported_seq":610,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-06-27T12:41:33.816085+0000","last_change":"2023-06-27T12:41:32.622065+0000","last_active":"2023-06-27T12:41:33.816085+0000","last_peered":"2023-06-27T12:41:33.816085+0000","last_clean":"2023-06-27T12:41:33.816085+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:33.816085+0000","last_undegraded":"2023-06-27T12:41:33.816085+0000","last_fullsized":"2023-06-27T12:41:33.816085+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6941357,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":179,"num_read_kb":118395,"num_write":133,"num_write_kb":38521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"184'134","reported_seq":504,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-06-27T12:41:33.813348+0000","last_change":"2023-06-27T12:41:32.622148+0000","last_active":"2023-06-27T12:41:33.813348+0000","last_peered":"2023-06-27T12:41:33.813348+0000","last_clean":"2023-06-27T12:41:33.813348+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:33.813348+0000","last_undegraded":"2023-06-27T12:41:33.813348+0000","last_fullsized":"2023-06-27T12:41:33.813348+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3546310,"num_objects":17,"num_object_clones":14,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":75,"num_read_kb":44588,"num_write":72,"num_write_kb":21091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"199'60","reported_seq":539,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.622232+0000","last_change":"2023-06-27T12:41:32.645648+0000","last_active":"2023-06-27T12:41:33.622232+0000","last_peered":"2023-06-27T12:41:33.622232+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:33.622232+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916064,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":324,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-06-27T12:41:33.632202+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:33.632202+0000","last_peered":"2023-06-27T12:41:33.632202+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:33.632202+0000","last_undegraded":"2023-06-27T12:41:33.632202+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":461,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-06-27T12:41:33.621308+0000","last_change":"2023-06-27T12:41:32.653740+0000","last_active":"2023-06-27T12:41:33.621308+0000","last_peered":"2023-06-27T12:41:33.621308+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:33.621308+0000","last_undegraded":"2023-06-27T12:41:33.621308+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":305,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-06-27T12:41:33.810161+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:33.810161+0000","last_peered":"2023-06-27T12:41:33.810161+0000","last_clean":"2023-06-27T12:41:33.810161+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:33.810161+0000","last_undegraded":"2023-06-27T12:41:33.810161+0000","last_fullsized":"2023-06-27T12:41:33.810161+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"188'101","reported_seq":586,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.625116+0000","last_change":"2023-06-27T12:41:32.621786+0000","last_active":"2023-06-27T12:41:33.625116+0000","last_peered":"2023-06-27T12:41:33.625116+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:33.625116+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6319465,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":56875,"num_write":64,"num_write_kb":17475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":13}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":308,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-06-27T12:41:33.626359+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:33.626359+0000","last_peered":"2023-06-27T12:41:33.626359+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:33.626359+0000","last_undegraded":"2023-06-27T12:41:33.626359+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"197'285","reported_seq":209,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.629254+0000","last_change":"2023-06-27T12:41:32.622989+0000","last_active":"2023-06-27T12:41:33.629254+0000","last_peered":"2023-06-27T12:41:33.629254+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:33.629254+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2554697,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":17,"num_read_kb":9352,"num_write":29,"num_write_kb":8559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":6}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":305,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-06-27T12:41:33.624399+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:33.624399+0000","last_peered":"2023-06-27T12:41:33.624399+0000","last_clean":"2023-06-27T12:41:33.624399+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:33.624399+0000","last_undegraded":"2023-06-27T12:41:33.624399+0000","last_fullsized":"2023-06-27T12:41:33.624399+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":57,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-06-27T12:41:33.804230+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:33.804230+0000","last_peered":"2023-06-27T12:41:33.804230+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:33.804230+0000","last_undegraded":"2023-06-27T12:41:33.804230+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"195'215","reported_seq":709,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.628439+0000","last_change":"2023-06-27T12:41:33.628853+0000","last_active":"2023-06-27T12:41:33.628439+0000","last_peered":"2023-06-27T12:41:33.628439+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:33.628439+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":15,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17496219,"num_objects":31,"num_object_clones":25,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":188,"num_read_kb":110398,"num_write":162,"num_write_kb":46946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":31}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":305,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-06-27T12:41:33.813165+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:33.813165+0000","last_peered":"2023-06-27T12:41:33.813165+0000","last_clean":"2023-06-27T12:41:33.813165+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:33.813165+0000","last_undegraded":"2023-06-27T12:41:33.813165+0000","last_fullsized":"2023-06-27T12:41:33.813165+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"195'112","reported_seq":178,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.619165+0000","last_change":"2023-06-27T12:41:32.652708+0000","last_active":"2023-06-27T12:41:33.619165+0000","last_peered":"2023-06-27T12:41:33.619165+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:33.619165+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":52,"num_read_kb":35679,"num_write":13,"num_write_kb":4215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":67,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-06-27T12:41:33.630279+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:33.630279+0000","last_peered":"2023-06-27T12:41:33.630279+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:33.630279+0000","last_undegraded":"2023-06-27T12:41:33.630279+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"194'116","reported_seq":611,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.628686+0000","last_change":"2023-06-27T12:41:32.618764+0000","last_active":"2023-06-27T12:41:33.628686+0000","last_peered":"2023-06-27T12:41:33.628686+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:33.628686+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9344791,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":70,"num_read_kb":71930,"num_write":82,"num_write_kb":24794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":16}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"184'67","reported_seq":170,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.622909+0000","last_change":"2023-06-27T12:41:32.651595+0000","last_active":"2023-06-27T12:41:33.622909+0000","last_peered":"2023-06-27T12:41:33.622909+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:33.622909+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":17010,"num_write":9,"num_write_kb":2650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":324,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-06-27T12:41:33.633141+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:33.633141+0000","last_peered":"2023-06-27T12:41:33.633141+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:33.633141+0000","last_undegraded":"2023-06-27T12:41:33.633141+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":305,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-06-27T12:41:33.620843+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:33.620843+0000","last_peered":"2023-06-27T12:41:33.620843+0000","last_clean":"2023-06-27T12:41:33.620843+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:33.620843+0000","last_undegraded":"2023-06-27T12:41:33.620843+0000","last_fullsized":"2023-06-27T12:41:33.620843+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"154'120","reported_seq":590,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:33.628682+0000","last_change":"2023-06-27T12:41:32.618312+0000","last_active":"2023-06-27T12:41:33.628682+0000","last_peered":"2023-06-27T12:41:33.628682+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:33.628682+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739640,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":60,"num_read_kb":56799,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":72909459,"num_objects":190,"num_object_clones":149,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":1061,"num_read_kb":754550,"num_write":898,"num_write_kb":262562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1826,"ondisk_log_size":1826,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167108,"kb_used_data":136188,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94204732,"statfs":{"total":96636764160,"available":96465645568,"internally_reserved":0,"allocated":139456512,"data_stored":161218995,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":187080,"kb_used_data":156160,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94184760,"statfs":{"total":96636764160,"available":96445194240,"internally_reserved":0,"allocated":159907840,"data_stored":187688458,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":202280,"kb_used_data":170360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94169560,"statfs":{"total":96636764160,"available":96429629440,"internally_reserved":0,"allocated":174448640,"data_stored":204653976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:37.809 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.c 3.8 3.2 3.1 3.4 3.7 2023-06-27T12:41:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:37.929+0000 7f9239dbb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:37.941+0000 7f9239dbb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:38.138 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c] on osd.0 to force-backfill; instructing pg(s) [3.1,3.2,3.7,3.8] on osd.1 to force-backfill; instructing pg(s) [3.4] on osd.2 to force-backfill; 2023-06-27T12:41:38.139 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:41:38.426 INFO:tasks.rados.rados.0.smithi138.stdout:761: done (3 left) 2023-06-27T12:41:38.426 INFO:tasks.rados.rados.0.smithi138.stdout:762: delete oid 5 current snap is 91 2023-06-27T12:41:38.426 INFO:tasks.rados.rados.0.smithi138.stdout:758: finishing write tid 1 to smithi138138488-26 2023-06-27T12:41:38.426 INFO:tasks.rados.rados.0.smithi138.stdout:758: finishing write tid 2 to smithi138138488-26 2023-06-27T12:41:38.427 INFO:tasks.rados.rados.0.smithi138.stdout:758: finishing write tid 3 to smithi138138488-26 2023-06-27T12:41:38.427 INFO:tasks.rados.rados.0.smithi138.stdout:758: finishing write tid 4 to smithi138138488-26 2023-06-27T12:41:38.427 INFO:tasks.rados.rados.0.smithi138.stdout:758: oid 26 updating version 0 to 122 2023-06-27T12:41:38.427 INFO:tasks.rados.rados.0.smithi138.stdout:758: oid 26 version 122 is already newer than 122 2023-06-27T12:41:38.427 INFO:tasks.rados.rados.0.smithi138.stdout:758: oid 26 version 122 is already newer than 122 2023-06-27T12:41:38.427 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 122 (ObjNum 360 snap 91 seq_num 360) dirty exists 2023-06-27T12:41:38.427 INFO:tasks.rados.rados.0.smithi138.stdout:758: left oid 26 (ObjNum 360 snap 91 seq_num 360) 2023-06-27T12:41:38.427 INFO:tasks.rados.rados.0.smithi138.stdout:760: finishing write tid 1 to smithi138138488-9 2023-06-27T12:41:38.428 INFO:tasks.rados.rados.0.smithi138.stdout:760: finishing write tid 2 to smithi138138488-9 2023-06-27T12:41:38.428 INFO:tasks.rados.rados.0.smithi138.stdout:760: finishing write tid 3 to smithi138138488-9 2023-06-27T12:41:38.428 INFO:tasks.rados.rados.0.smithi138.stdout:760: finishing write tid 4 to smithi138138488-9 2023-06-27T12:41:38.428 INFO:tasks.rados.rados.0.smithi138.stdout:760: oid 9 updating version 0 to 227 2023-06-27T12:41:38.428 INFO:tasks.rados.rados.0.smithi138.stdout:760: oid 9 version 227 is already newer than 224 2023-06-27T12:41:38.428 INFO:tasks.rados.rados.0.smithi138.stdout:760: oid 9 version 227 is already newer than 227 2023-06-27T12:41:38.429 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 227 (ObjNum 361 snap 91 seq_num 361) dirty exists 2023-06-27T12:41:38.429 INFO:tasks.rados.rados.0.smithi138.stdout:760: left oid 9 (ObjNum 361 snap 91 seq_num 361) 2023-06-27T12:41:38.493 INFO:tasks.rados.rados.0.smithi138.stdout:758: done (3 left) 2023-06-27T12:41:38.493 INFO:tasks.rados.rados.0.smithi138.stdout:759: done (2 left) 2023-06-27T12:41:38.494 INFO:tasks.rados.rados.0.smithi138.stdout:760: done (1 left) 2023-06-27T12:41:38.494 INFO:tasks.rados.rados.0.smithi138.stdout:762: done (0 left) 2023-06-27T12:41:38.494 INFO:tasks.rados.rados.0.smithi138.stdout:763: write (excl) oid 16 current snap is 91 2023-06-27T12:41:38.494 INFO:tasks.rados.rados.0.smithi138.stdout:763: seq_num 363 ranges {500941=783978,1942140=158801} 2023-06-27T12:41:38.501 INFO:tasks.rados.rados.0.smithi138.stdout:763: writing smithi138138488-16 from 500941 to 1284919 tid 1 2023-06-27T12:41:38.506 INFO:tasks.rados.rados.0.smithi138.stdout:763: writing smithi138138488-16 from 1942140 to 2100941 tid 2 2023-06-27T12:41:38.506 INFO:tasks.rados.rados.0.smithi138.stdout:764: write oid 31 current snap is 91 2023-06-27T12:41:38.506 INFO:tasks.rados.rados.0.smithi138.stdout:764: seq_num 364 ranges {517748=459227,1600622=406110} 2023-06-27T12:41:38.511 INFO:tasks.rados.rados.0.smithi138.stdout:764: writing smithi138138488-31 from 517748 to 976975 tid 1 2023-06-27T12:41:38.517 INFO:tasks.rados.rados.0.smithi138.stdout:764: writing smithi138138488-31 from 1600622 to 2006732 tid 2 2023-06-27T12:41:38.517 INFO:tasks.rados.rados.0.smithi138.stdout:765: write (excl) oid 30 current snap is 91 2023-06-27T12:41:38.517 INFO:tasks.rados.rados.0.smithi138.stdout:765: seq_num 365 ranges {534555=650829,1659104=475451} 2023-06-27T12:41:38.524 INFO:tasks.rados.rados.0.smithi138.stdout:765: writing smithi138138488-30 from 534555 to 1185384 tid 1 2023-06-27T12:41:38.529 INFO:tasks.rados.rados.0.smithi138.stdout:765: writing smithi138138488-30 from 1659104 to 2134555 tid 2 2023-06-27T12:41:38.529 INFO:tasks.rados.rados.0.smithi138.stdout:766: write (excl) oid 35 current snap is 91 2023-06-27T12:41:38.530 INFO:tasks.rados.rados.0.smithi138.stdout:766: seq_num 366 ranges {551362=726078,1717586=409779} 2023-06-27T12:41:38.537 INFO:tasks.rados.rados.0.smithi138.stdout:766: writing smithi138138488-35 from 551362 to 1277440 tid 1 2023-06-27T12:41:38.542 INFO:tasks.rados.rados.0.smithi138.stdout:766: writing smithi138138488-35 from 1717586 to 2127365 tid 2 2023-06-27T12:41:38.543 INFO:tasks.rados.rados.0.smithi138.stdout:767: write (excl) oid 10 current snap is 91 2023-06-27T12:41:38.543 INFO:tasks.rados.rados.0.smithi138.stdout:767: seq_num 367 ranges {568169=401327,1376068=669790} 2023-06-27T12:41:38.548 INFO:tasks.rados.rados.0.smithi138.stdout:767: writing smithi138138488-10 from 568169 to 969496 tid 1 2023-06-27T12:41:38.554 INFO:tasks.rados.rados.0.smithi138.stdout:767: writing smithi138138488-10 from 1376068 to 2045858 tid 2 2023-06-27T12:41:38.554 INFO:tasks.rados.rados.0.smithi138.stdout:768: write (excl) oid 15 current snap is 91 2023-06-27T12:41:38.554 INFO:tasks.rados.rados.0.smithi138.stdout:768: seq_num 368 ranges {584976=476576,1834550=350426} 2023-06-27T12:41:38.560 INFO:tasks.rados.rados.0.smithi138.stdout:768: writing smithi138138488-15 from 584976 to 1061552 tid 1 2023-06-27T12:41:38.567 INFO:tasks.rados.rados.0.smithi138.stdout:768: writing smithi138138488-15 from 1834550 to 2184976 tid 2 2023-06-27T12:41:38.567 INFO:tasks.rados.rados.0.smithi138.stdout:769: write (excl) oid 7 current snap is 91 2023-06-27T12:41:38.567 INFO:tasks.rados.rados.0.smithi138.stdout:763: finishing write tid 1 to smithi138138488-16 2023-06-27T12:41:38.568 INFO:tasks.rados.rados.0.smithi138.stdout:769: seq_num 369 ranges {601783=551825,1776679=425104} 2023-06-27T12:41:38.574 INFO:tasks.rados.rados.0.smithi138.stdout:769: writing smithi138138488-7 from 601783 to 1153608 tid 1 2023-06-27T12:41:38.581 INFO:tasks.rados.rados.0.smithi138.stdout:769: writing smithi138138488-7 from 1776679 to 2201783 tid 2 2023-06-27T12:41:38.581 INFO:tasks.rados.rados.0.smithi138.stdout:770: read oid 49 snap -1 2023-06-27T12:41:38.581 INFO:tasks.rados.rados.0.smithi138.stdout:770: expect (ObjNum 245 snap 68 seq_num 245) 2023-06-27T12:41:38.581 INFO:tasks.rados.rados.0.smithi138.stdout:771: write oid 27 current snap is 91 2023-06-27T12:41:38.581 INFO:tasks.rados.rados.0.smithi138.stdout:771: seq_num 370 ranges {618590=627074,1835161=383429} 2023-06-27T12:41:38.588 INFO:tasks.rados.rados.0.smithi138.stdout:771: writing smithi138138488-27 from 618590 to 1245664 tid 1 2023-06-27T12:41:38.594 INFO:tasks.rados.rados.0.smithi138.stdout:771: writing smithi138138488-27 from 1835161 to 2218590 tid 2 2023-06-27T12:41:38.594 INFO:tasks.rados.rados.0.smithi138.stdout:772: read oid 20 snap -1 2023-06-27T12:41:38.595 INFO:tasks.rados.rados.0.smithi138.stdout:772: expect deleted 2023-06-27T12:41:38.595 INFO:tasks.rados.rados.0.smithi138.stdout:773: read oid 25 snap -1 2023-06-27T12:41:38.595 INFO:tasks.rados.rados.0.smithi138.stdout:773: expect (ObjNum 356 snap 90 seq_num 356) 2023-06-27T12:41:38.595 INFO:tasks.rados.rados.0.smithi138.stdout:774: read oid 3 snap -1 2023-06-27T12:41:38.595 INFO:tasks.rados.rados.0.smithi138.stdout:774: expect deleted 2023-06-27T12:41:38.595 INFO:tasks.rados.rados.0.smithi138.stdout:775: read oid 13 snap -1 2023-06-27T12:41:38.595 INFO:tasks.rados.rados.0.smithi138.stdout:775: expect (ObjNum 338 snap 84 seq_num 338) 2023-06-27T12:41:38.595 INFO:tasks.rados.rados.0.smithi138.stdout:776: write oid 38 current snap is 91 2023-06-27T12:41:38.596 INFO:tasks.rados.rados.0.smithi138.stdout:776: seq_num 371 ranges {635397=702323,1893643=341754} 2023-06-27T12:41:38.604 INFO:tasks.rados.rados.0.smithi138.stdout:776: writing smithi138138488-38 from 635397 to 1337720 tid 1 2023-06-27T12:41:38.610 INFO:tasks.rados.rados.0.smithi138.stdout:776: writing smithi138138488-38 from 1893643 to 2235397 tid 2 2023-06-27T12:41:38.610 INFO:tasks.rados.rados.0.smithi138.stdout:777: write oid 19 current snap is 91 2023-06-27T12:41:38.610 INFO:tasks.rados.rados.0.smithi138.stdout:777: seq_num 372 ranges {652204=777572,1952125=300079} 2023-06-27T12:41:38.619 INFO:tasks.rados.rados.0.smithi138.stdout:777: writing smithi138138488-19 from 652204 to 1429776 tid 1 2023-06-27T12:41:38.624 INFO:tasks.rados.rados.0.smithi138.stdout:777: writing smithi138138488-19 from 1952125 to 2252204 tid 2 2023-06-27T12:41:38.624 INFO:tasks.rados.rados.0.smithi138.stdout:778: write (excl) oid 23 current snap is 91 2023-06-27T12:41:38.624 INFO:tasks.rados.rados.0.smithi138.stdout:770: expect (ObjNum 245 snap 68 seq_num 245) 2023-06-27T12:41:38.663 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:38.749 INFO:tasks.rados.rados.0.smithi138.stdout:766: finishing write tid 1 to smithi138138488-35 2023-06-27T12:41:38.749 INFO:tasks.rados.rados.0.smithi138.stdout:763: finishing write tid 2 to smithi138138488-16 2023-06-27T12:41:38.749 INFO:tasks.rados.rados.0.smithi138.stdout:763: finishing write tid 3 to smithi138138488-16 2023-06-27T12:41:38.750 INFO:tasks.rados.rados.0.smithi138.stdout:773: expect (ObjNum 356 snap 90 seq_num 356) 2023-06-27T12:41:38.815 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:38.810+0000 7faff75f4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:38.818 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:38.814+0000 7faff75f4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:38.874 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:38.876 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:38.877 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:38.914 INFO:tasks.rados.rados.0.smithi138.stdout:765: finishing write tid 1 to smithi138138488-30 2023-06-27T12:41:38.915 INFO:tasks.rados.rados.0.smithi138.stdout:778: seq_num 373 ranges {669011=569174,2010607=258404} 2023-06-27T12:41:38.921 INFO:tasks.rados.rados.0.smithi138.stdout:778: writing smithi138138488-23 from 669011 to 1238185 tid 1 2023-06-27T12:41:38.927 INFO:tasks.rados.rados.0.smithi138.stdout:778: writing smithi138138488-23 from 2010607 to 2269011 tid 2 2023-06-27T12:41:38.927 INFO:tasks.rados.rados.0.smithi138.stdout:770: done (15 left) 2023-06-27T12:41:38.927 INFO:tasks.rados.rados.0.smithi138.stdout:773: done (14 left) 2023-06-27T12:41:38.928 INFO:tasks.rados.rados.0.smithi138.stdout:779: write (excl) oid 32 current snap is 91 2023-06-27T12:41:38.928 INFO:tasks.rados.rados.0.smithi138.stdout:779: seq_num 374 ranges {685818=644423,2069089=216729} 2023-06-27T12:41:38.934 INFO:tasks.rados.rados.0.smithi138.stdout:779: writing smithi138138488-32 from 685818 to 1330241 tid 1 2023-06-27T12:41:38.940 INFO:tasks.rados.rados.0.smithi138.stdout:779: writing smithi138138488-32 from 2069089 to 2285818 tid 2 2023-06-27T12:41:38.940 INFO:tasks.rados.rados.0.smithi138.stdout:780: write oid 4 current snap is 91 2023-06-27T12:41:38.940 INFO:tasks.rados.rados.0.smithi138.stdout:763: finishing write tid 4 to smithi138138488-16 2023-06-27T12:41:38.941 INFO:tasks.rados.rados.0.smithi138.stdout:763: oid 16 updating version 0 to 261 2023-06-27T12:41:38.941 INFO:tasks.rados.rados.0.smithi138.stdout:763: oid 16 version 261 is already newer than 261 2023-06-27T12:41:38.941 INFO:tasks.rados.rados.0.smithi138.stdout:763: oid 16 version 261 is already newer than 259 2023-06-27T12:41:38.942 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 261 (ObjNum 363 snap 91 seq_num 363) dirty exists 2023-06-27T12:41:38.942 INFO:tasks.rados.rados.0.smithi138.stdout:763: left oid 16 (ObjNum 363 snap 91 seq_num 363) 2023-06-27T12:41:38.942 INFO:tasks.rados.rados.0.smithi138.stdout:766: finishing write tid 2 to smithi138138488-35 2023-06-27T12:41:38.942 INFO:tasks.rados.rados.0.smithi138.stdout:768: finishing write tid 1 to smithi138138488-15 2023-06-27T12:41:38.943 INFO:tasks.rados.rados.0.smithi138.stdout:766: finishing write tid 3 to smithi138138488-35 2023-06-27T12:41:38.943 INFO:tasks.rados.rados.0.smithi138.stdout:766: finishing write tid 4 to smithi138138488-35 2023-06-27T12:41:38.943 INFO:tasks.rados.rados.0.smithi138.stdout:766: oid 35 updating version 0 to 295 2023-06-27T12:41:38.943 INFO:tasks.rados.rados.0.smithi138.stdout:766: oid 35 version 295 is already newer than 295 2023-06-27T12:41:38.944 INFO:tasks.rados.rados.0.smithi138.stdout:766: oid 35 version 295 is already newer than 293 2023-06-27T12:41:38.944 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 295 (ObjNum 366 snap 91 seq_num 366) dirty exists 2023-06-27T12:41:38.944 INFO:tasks.rados.rados.0.smithi138.stdout:766: left oid 35 (ObjNum 366 snap 91 seq_num 366) 2023-06-27T12:41:38.944 INFO:tasks.rados.rados.0.smithi138.stdout:780: seq_num 375 ranges {702625=719672,2127571=175054} 2023-06-27T12:41:38.947 INFO:tasks.rados.rados.0.smithi138.stdout:780: writing smithi138138488-4 from 702625 to 1422297 tid 1 2023-06-27T12:41:38.952 INFO:tasks.rados.rados.0.smithi138.stdout:780: writing smithi138138488-4 from 2127571 to 2302625 tid 2 2023-06-27T12:41:38.953 INFO:tasks.rados.rados.0.smithi138.stdout:763: done (15 left) 2023-06-27T12:41:38.953 INFO:tasks.rados.rados.0.smithi138.stdout:766: done (14 left) 2023-06-27T12:41:38.953 INFO:tasks.rados.rados.0.smithi138.stdout:781: delete oid 12 current snap is 91 2023-06-27T12:41:38.954 INFO:tasks.rados.rados.0.smithi138.stdout:775: expect (ObjNum 338 snap 84 seq_num 338) 2023-06-27T12:41:39.098 INFO:tasks.rados.rados.0.smithi138.stdout:767: finishing write tid 1 to smithi138138488-10 2023-06-27T12:41:39.099 INFO:tasks.rados.rados.0.smithi138.stdout:771: finishing write tid 1 to smithi138138488-27 2023-06-27T12:41:39.099 INFO:tasks.rados.rados.0.smithi138.stdout:768: finishing write tid 2 to smithi138138488-15 2023-06-27T12:41:39.099 INFO:tasks.rados.rados.0.smithi138.stdout:765: finishing write tid 2 to smithi138138488-30 2023-06-27T12:41:39.100 INFO:tasks.rados.rados.0.smithi138.stdout:768: finishing write tid 3 to smithi138138488-15 2023-06-27T12:41:39.100 INFO:tasks.rados.rados.0.smithi138.stdout:774: done (14 left) 2023-06-27T12:41:39.100 INFO:tasks.rados.rados.0.smithi138.stdout:775: done (13 left) 2023-06-27T12:41:39.100 INFO:tasks.rados.rados.0.smithi138.stdout:781: done (12 left) 2023-06-27T12:41:39.101 INFO:tasks.rados.rados.0.smithi138.stdout:782: delete oid 6 current snap is 91 2023-06-27T12:41:39.101 INFO:tasks.rados.rados.0.smithi138.stdout:768: finishing write tid 4 to smithi138138488-15 2023-06-27T12:41:39.101 INFO:tasks.rados.rados.0.smithi138.stdout:768: oid 15 updating version 0 to 40 2023-06-27T12:41:39.101 INFO:tasks.rados.rados.0.smithi138.stdout:768: oid 15 updating version 40 to 42 2023-06-27T12:41:39.102 INFO:tasks.rados.rados.0.smithi138.stdout:768: oid 15 version 42 is already newer than 42 2023-06-27T12:41:39.102 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 42 (ObjNum 368 snap 91 seq_num 368) dirty exists 2023-06-27T12:41:39.102 INFO:tasks.rados.rados.0.smithi138.stdout:768: left oid 15 (ObjNum 368 snap 91 seq_num 368) 2023-06-27T12:41:39.103 INFO:tasks.rados.rados.0.smithi138.stdout:764: finishing write tid 1 to smithi138138488-31 2023-06-27T12:41:39.103 INFO:tasks.rados.rados.0.smithi138.stdout:764: finishing write tid 2 to smithi138138488-31 2023-06-27T12:41:39.103 INFO:tasks.rados.rados.0.smithi138.stdout:764: finishing write tid 3 to smithi138138488-31 2023-06-27T12:41:39.104 INFO:tasks.rados.rados.0.smithi138.stdout:771: finishing write tid 2 to smithi138138488-27 2023-06-27T12:41:39.104 INFO:tasks.rados.rados.0.smithi138.stdout:771: finishing write tid 3 to smithi138138488-27 2023-06-27T12:41:39.104 INFO:tasks.rados.rados.0.smithi138.stdout:764: finishing write tid 4 to smithi138138488-31 2023-06-27T12:41:39.104 INFO:tasks.rados.rados.0.smithi138.stdout:764: oid 31 updating version 0 to 264 2023-06-27T12:41:39.104 INFO:tasks.rados.rados.0.smithi138.stdout:764: oid 31 version 264 is already newer than 264 2023-06-27T12:41:39.105 INFO:tasks.rados.rados.0.smithi138.stdout:764: oid 31 version 264 is already newer than 264 2023-06-27T12:41:39.105 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 264 (ObjNum 364 snap 91 seq_num 364) dirty exists 2023-06-27T12:41:39.105 INFO:tasks.rados.rados.0.smithi138.stdout:764: left oid 31 (ObjNum 364 snap 91 seq_num 364) 2023-06-27T12:41:39.106 INFO:tasks.rados.rados.0.smithi138.stdout:771: finishing write tid 4 to smithi138138488-27 2023-06-27T12:41:39.106 INFO:tasks.rados.rados.0.smithi138.stdout:771: oid 27 updating version 0 to 115 2023-06-27T12:41:39.106 INFO:tasks.rados.rados.0.smithi138.stdout:771: oid 27 version 115 is already newer than 115 2023-06-27T12:41:39.107 INFO:tasks.rados.rados.0.smithi138.stdout:771: oid 27 version 115 is already newer than 115 2023-06-27T12:41:39.107 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 115 (ObjNum 370 snap 91 seq_num 370) dirty exists 2023-06-27T12:41:39.107 INFO:tasks.rados.rados.0.smithi138.stdout:771: left oid 27 (ObjNum 370 snap 91 seq_num 370) 2023-06-27T12:41:39.107 INFO:tasks.rados.rados.0.smithi138.stdout:767: finishing write tid 2 to smithi138138488-10 2023-06-27T12:41:39.108 INFO:tasks.rados.rados.0.smithi138.stdout:767: finishing write tid 3 to smithi138138488-10 2023-06-27T12:41:39.108 INFO:tasks.rados.rados.0.smithi138.stdout:765: finishing write tid 3 to smithi138138488-30 2023-06-27T12:41:39.108 INFO:tasks.rados.rados.0.smithi138.stdout:765: finishing write tid 4 to smithi138138488-30 2023-06-27T12:41:39.108 INFO:tasks.rados.rados.0.smithi138.stdout:765: oid 30 updating version 0 to 126 2023-06-27T12:41:39.109 INFO:tasks.rados.rados.0.smithi138.stdout:765: oid 30 version 126 is already newer than 126 2023-06-27T12:41:39.109 INFO:tasks.rados.rados.0.smithi138.stdout:765: oid 30 version 126 is already newer than 124 2023-06-27T12:41:39.109 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 126 (ObjNum 365 snap 91 seq_num 365) dirty exists 2023-06-27T12:41:39.109 INFO:tasks.rados.rados.0.smithi138.stdout:765: left oid 30 (ObjNum 365 snap 91 seq_num 365) 2023-06-27T12:41:39.110 INFO:tasks.rados.rados.0.smithi138.stdout:777: finishing write tid 1 to smithi138138488-19 2023-06-27T12:41:39.110 INFO:tasks.rados.rados.0.smithi138.stdout:777: finishing write tid 2 to smithi138138488-19 2023-06-27T12:41:39.110 INFO:tasks.rados.rados.0.smithi138.stdout:767: finishing write tid 4 to smithi138138488-10 2023-06-27T12:41:39.110 INFO:tasks.rados.rados.0.smithi138.stdout:767: oid 10 updating version 0 to 139 2023-06-27T12:41:39.111 INFO:tasks.rados.rados.0.smithi138.stdout:767: oid 10 version 139 is already newer than 137 2023-06-27T12:41:39.111 INFO:tasks.rados.rados.0.smithi138.stdout:767: oid 10 version 139 is already newer than 139 2023-06-27T12:41:39.111 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 139 (ObjNum 367 snap 91 seq_num 367) dirty exists 2023-06-27T12:41:39.111 INFO:tasks.rados.rados.0.smithi138.stdout:767: left oid 10 (ObjNum 367 snap 91 seq_num 367) 2023-06-27T12:41:39.112 INFO:tasks.rados.rados.0.smithi138.stdout:776: finishing write tid 1 to smithi138138488-38 2023-06-27T12:41:39.112 INFO:tasks.rados.rados.0.smithi138.stdout:776: finishing write tid 2 to smithi138138488-38 2023-06-27T12:41:39.112 INFO:tasks.rados.rados.0.smithi138.stdout:777: finishing write tid 3 to smithi138138488-19 2023-06-27T12:41:39.112 INFO:tasks.rados.rados.0.smithi138.stdout:777: finishing write tid 4 to smithi138138488-19 2023-06-27T12:41:39.113 INFO:tasks.rados.rados.0.smithi138.stdout:777: oid 19 updating version 0 to 232 2023-06-27T12:41:39.113 INFO:tasks.rados.rados.0.smithi138.stdout:777: oid 19 version 232 is already newer than 232 2023-06-27T12:41:39.113 INFO:tasks.rados.rados.0.smithi138.stdout:777: oid 19 version 232 is already newer than 232 2023-06-27T12:41:39.114 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 232 (ObjNum 372 snap 91 seq_num 372) dirty exists 2023-06-27T12:41:39.114 INFO:tasks.rados.rados.0.smithi138.stdout:777: left oid 19 (ObjNum 372 snap 91 seq_num 372) 2023-06-27T12:41:39.114 INFO:tasks.rados.rados.0.smithi138.stdout:776: finishing write tid 3 to smithi138138488-38 2023-06-27T12:41:39.115 INFO:tasks.rados.rados.0.smithi138.stdout:769: finishing write tid 1 to smithi138138488-7 2023-06-27T12:41:39.115 INFO:tasks.rados.rados.0.smithi138.stdout:776: finishing write tid 4 to smithi138138488-38 2023-06-27T12:41:39.115 INFO:tasks.rados.rados.0.smithi138.stdout:776: oid 38 updating version 0 to 10 2023-06-27T12:41:39.115 INFO:tasks.rados.rados.0.smithi138.stdout:776: oid 38 version 10 is already newer than 10 2023-06-27T12:41:39.116 INFO:tasks.rados.rados.0.smithi138.stdout:776: oid 38 version 10 is already newer than 10 2023-06-27T12:41:39.116 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 10 (ObjNum 371 snap 91 seq_num 371) dirty exists 2023-06-27T12:41:39.116 INFO:tasks.rados.rados.0.smithi138.stdout:776: left oid 38 (ObjNum 371 snap 91 seq_num 371) 2023-06-27T12:41:39.117 INFO:tasks.rados.rados.0.smithi138.stdout:769: finishing write tid 2 to smithi138138488-7 2023-06-27T12:41:39.117 INFO:tasks.rados.rados.0.smithi138.stdout:769: finishing write tid 3 to smithi138138488-7 2023-06-27T12:41:39.117 INFO:tasks.rados.rados.0.smithi138.stdout:769: finishing write tid 4 to smithi138138488-7 2023-06-27T12:41:39.117 INFO:tasks.rados.rados.0.smithi138.stdout:769: oid 7 updating version 0 to 142 2023-06-27T12:41:39.118 INFO:tasks.rados.rados.0.smithi138.stdout:769: oid 7 version 142 is already newer than 142 2023-06-27T12:41:39.118 INFO:tasks.rados.rados.0.smithi138.stdout:769: oid 7 version 142 is already newer than 140 2023-06-27T12:41:39.118 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 142 (ObjNum 369 snap 91 seq_num 369) dirty exists 2023-06-27T12:41:39.118 INFO:tasks.rados.rados.0.smithi138.stdout:769: left oid 7 (ObjNum 369 snap 91 seq_num 369) 2023-06-27T12:41:39.119 INFO:tasks.rados.rados.0.smithi138.stdout:778: finishing write tid 1 to smithi138138488-23 2023-06-27T12:41:39.119 INFO:tasks.rados.rados.0.smithi138.stdout:780: finishing write tid 1 to smithi138138488-4 2023-06-27T12:41:39.119 INFO:tasks.rados.rados.0.smithi138.stdout:779: finishing write tid 1 to smithi138138488-32 2023-06-27T12:41:39.119 INFO:tasks.rados.rados.0.smithi138.stdout:780: finishing write tid 2 to smithi138138488-4 2023-06-27T12:41:39.120 INFO:tasks.rados.rados.0.smithi138.stdout:780: finishing write tid 3 to smithi138138488-4 2023-06-27T12:41:39.120 INFO:tasks.rados.rados.0.smithi138.stdout:780: finishing write tid 4 to smithi138138488-4 2023-06-27T12:41:39.120 INFO:tasks.rados.rados.0.smithi138.stdout:780: oid 4 updating version 0 to 128 2023-06-27T12:41:39.120 INFO:tasks.rados.rados.0.smithi138.stdout:780: oid 4 updating version 128 to 129 2023-06-27T12:41:39.121 INFO:tasks.rados.rados.0.smithi138.stdout:780: oid 4 version 129 is already newer than 129 2023-06-27T12:41:39.121 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 129 (ObjNum 375 snap 91 seq_num 375) dirty exists 2023-06-27T12:41:39.121 INFO:tasks.rados.rados.0.smithi138.stdout:780: left oid 4 (ObjNum 375 snap 91 seq_num 375) 2023-06-27T12:41:39.121 INFO:tasks.rados.rados.0.smithi138.stdout:778: finishing write tid 2 to smithi138138488-23 2023-06-27T12:41:39.122 INFO:tasks.rados.rados.0.smithi138.stdout:778: finishing write tid 3 to smithi138138488-23 2023-06-27T12:41:39.122 INFO:tasks.rados.rados.0.smithi138.stdout:778: finishing write tid 4 to smithi138138488-23 2023-06-27T12:41:39.122 INFO:tasks.rados.rados.0.smithi138.stdout:778: oid 23 updating version 0 to 108 2023-06-27T12:41:39.122 INFO:tasks.rados.rados.0.smithi138.stdout:778: oid 23 version 108 is already newer than 106 2023-06-27T12:41:39.123 INFO:tasks.rados.rados.0.smithi138.stdout:778: oid 23 version 108 is already newer than 108 2023-06-27T12:41:39.123 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 108 (ObjNum 373 snap 91 seq_num 373) dirty exists 2023-06-27T12:41:39.123 INFO:tasks.rados.rados.0.smithi138.stdout:778: left oid 23 (ObjNum 373 snap 91 seq_num 373) 2023-06-27T12:41:39.123 INFO:tasks.rados.rados.0.smithi138.stdout:779: finishing write tid 2 to smithi138138488-32 2023-06-27T12:41:39.124 INFO:tasks.rados.rados.0.smithi138.stdout:779: finishing write tid 3 to smithi138138488-32 2023-06-27T12:41:39.124 INFO:tasks.rados.rados.0.smithi138.stdout:779: finishing write tid 4 to smithi138138488-32 2023-06-27T12:41:39.124 INFO:tasks.rados.rados.0.smithi138.stdout:779: oid 32 updating version 0 to 300 2023-06-27T12:41:39.125 INFO:tasks.rados.rados.0.smithi138.stdout:779: oid 32 version 300 is already newer than 300 2023-06-27T12:41:39.125 INFO:tasks.rados.rados.0.smithi138.stdout:779: oid 32 version 300 is already newer than 298 2023-06-27T12:41:39.125 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 300 (ObjNum 374 snap 91 seq_num 374) dirty exists 2023-06-27T12:41:39.126 INFO:tasks.rados.rados.0.smithi138.stdout:779: left oid 32 (ObjNum 374 snap 91 seq_num 374) 2023-06-27T12:41:39.141 INFO:tasks.rados.rados.0.smithi138.stdout:764: done (12 left) 2023-06-27T12:41:39.141 INFO:tasks.rados.rados.0.smithi138.stdout:765: done (11 left) 2023-06-27T12:41:39.141 INFO:tasks.rados.rados.0.smithi138.stdout:767: done (10 left) 2023-06-27T12:41:39.142 INFO:tasks.rados.rados.0.smithi138.stdout:768: done (9 left) 2023-06-27T12:41:39.142 INFO:tasks.rados.rados.0.smithi138.stdout:769: done (8 left) 2023-06-27T12:41:39.142 INFO:tasks.rados.rados.0.smithi138.stdout:771: done (7 left) 2023-06-27T12:41:39.142 INFO:tasks.rados.rados.0.smithi138.stdout:772: done (6 left) 2023-06-27T12:41:39.143 INFO:tasks.rados.rados.0.smithi138.stdout:776: done (5 left) 2023-06-27T12:41:39.143 INFO:tasks.rados.rados.0.smithi138.stdout:777: done (4 left) 2023-06-27T12:41:39.143 INFO:tasks.rados.rados.0.smithi138.stdout:778: done (3 left) 2023-06-27T12:41:39.143 INFO:tasks.rados.rados.0.smithi138.stdout:779: done (2 left) 2023-06-27T12:41:39.144 INFO:tasks.rados.rados.0.smithi138.stdout:780: done (1 left) 2023-06-27T12:41:39.144 INFO:tasks.rados.rados.0.smithi138.stdout:782: done (0 left) 2023-06-27T12:41:39.144 INFO:tasks.rados.rados.0.smithi138.stdout:783: delete oid 10 current snap is 91 2023-06-27T12:41:39.186 INFO:tasks.rados.rados.0.smithi138.stdout:783: done (0 left) 2023-06-27T12:41:39.186 INFO:tasks.rados.rados.0.smithi138.stdout:784: read oid 36 snap -1 2023-06-27T12:41:39.186 INFO:tasks.rados.rados.0.smithi138.stdout:784: expect (ObjNum 263 snap 70 seq_num 263) 2023-06-27T12:41:39.187 INFO:tasks.rados.rados.0.smithi138.stdout:785: write oid 24 current snap is 91 2023-06-27T12:41:39.187 INFO:tasks.rados.rados.0.smithi138.stdout:785: seq_num 379 ranges {769853=620668,1845146=524707} 2023-06-27T12:41:39.193 INFO:tasks.rados.rados.0.smithi138.stdout:785: writing smithi138138488-24 from 769853 to 1390521 tid 1 2023-06-27T12:41:39.200 INFO:tasks.rados.rados.0.smithi138.stdout:785: writing smithi138138488-24 from 1845146 to 2369853 tid 2 2023-06-27T12:41:39.200 INFO:tasks.rados.rados.0.smithi138.stdout:786: read oid 28 snap -1 2023-06-27T12:41:39.201 INFO:tasks.rados.rados.0.smithi138.stdout:786: expect (ObjNum 330 snap 83 seq_num 330) 2023-06-27T12:41:39.201 INFO:tasks.rados.rados.0.smithi138.stdout:787: snap_create 2023-06-27T12:41:39.212 INFO:tasks.rados.rados.0.smithi138.stdout:784: expect (ObjNum 263 snap 70 seq_num 263) 2023-06-27T12:41:39.346 INFO:tasks.rados.rados.0.smithi138.stdout:786: expect (ObjNum 330 snap 83 seq_num 330) 2023-06-27T12:41:39.616 INFO:tasks.rados.rados.0.smithi138.stdout:785: finishing write tid 1 to smithi138138488-24 2023-06-27T12:41:39.617 INFO:tasks.rados.rados.0.smithi138.stdout:785: finishing write tid 2 to smithi138138488-24 2023-06-27T12:41:39.617 INFO:tasks.rados.rados.0.smithi138.stdout:785: finishing write tid 3 to smithi138138488-24 2023-06-27T12:41:39.617 INFO:tasks.rados.rados.0.smithi138.stdout:784: done (3 left) 2023-06-27T12:41:39.617 INFO:tasks.rados.rados.0.smithi138.stdout:786: done (2 left) 2023-06-27T12:41:39.618 INFO:tasks.rados.rados.0.smithi138.stdout:787: done (1 left) 2023-06-27T12:41:39.618 INFO:tasks.rados.rados.0.smithi138.stdout:788: snap_create 2023-06-27T12:41:39.618 INFO:tasks.rados.rados.0.smithi138.stdout:785: finishing write tid 4 to smithi138138488-24 2023-06-27T12:41:39.618 INFO:tasks.rados.rados.0.smithi138.stdout:785: oid 24 updating version 0 to 267 2023-06-27T12:41:39.619 INFO:tasks.rados.rados.0.smithi138.stdout:785: oid 24 version 267 is already newer than 267 2023-06-27T12:41:39.619 INFO:tasks.rados.rados.0.smithi138.stdout:785: oid 24 version 267 is already newer than 267 2023-06-27T12:41:39.619 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 267 (ObjNum 379 snap 91 seq_num 379) dirty exists 2023-06-27T12:41:39.619 INFO:tasks.rados.rados.0.smithi138.stdout:785: left oid 24 (ObjNum 379 snap 91 seq_num 379) 2023-06-27T12:41:40.435 INFO:tasks.rados.rados.0.smithi138.stdout:785: done (1 left) 2023-06-27T12:41:40.435 INFO:tasks.rados.rados.0.smithi138.stdout:788: done (0 left) 2023-06-27T12:41:40.435 INFO:tasks.rados.rados.0.smithi138.stdout:789: snap_remove snap 91 2023-06-27T12:41:40.879 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:41.005 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:41.000+0000 7fbbf14ab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:41.008 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:41.004+0000 7fbbf14ab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:41.054 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:41.056 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:41.057 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:41.355 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~66s 2023-06-27T12:41:41.436 INFO:tasks.rados.rados.0.smithi138.stdout:789: done (0 left) 2023-06-27T12:41:41.437 INFO:tasks.rados.rados.0.smithi138.stdout:790: write oid 9 current snap is 93 2023-06-27T12:41:41.437 INFO:tasks.rados.rados.0.smithi138.stdout:790: seq_num 380 ranges {786660=695917,1903628=435462} 2023-06-27T12:41:41.445 INFO:tasks.rados.rados.0.smithi138.stdout:790: writing smithi138138488-9 from 786660 to 1482577 tid 1 2023-06-27T12:41:41.451 INFO:tasks.rados.rados.0.smithi138.stdout:790: writing smithi138138488-9 from 1903628 to 2339090 tid 2 2023-06-27T12:41:41.451 INFO:tasks.rados.rados.0.smithi138.stdout:791: read oid 33 snap -1 2023-06-27T12:41:41.451 INFO:tasks.rados.rados.0.smithi138.stdout:791: expect (ObjNum 331 snap 83 seq_num 331) 2023-06-27T12:41:41.451 INFO:tasks.rados.rados.0.smithi138.stdout:792: read oid 31 snap -1 2023-06-27T12:41:41.452 INFO:tasks.rados.rados.0.smithi138.stdout:792: expect (ObjNum 364 snap 91 seq_num 364) 2023-06-27T12:41:41.452 INFO:tasks.rados.rados.0.smithi138.stdout:793: write oid 29 current snap is 93 2023-06-27T12:41:41.452 INFO:tasks.rados.rados.0.smithi138.stdout:793: seq_num 381 ranges {403467=487519,1562110=579120} 2023-06-27T12:41:41.457 INFO:tasks.rados.rados.0.smithi138.stdout:793: writing smithi138138488-29 from 403467 to 890986 tid 1 2023-06-27T12:41:41.466 INFO:tasks.rados.rados.0.smithi138.stdout:793: writing smithi138138488-29 from 1562110 to 2141230 tid 2 2023-06-27T12:41:41.466 INFO:tasks.rados.rados.0.smithi138.stdout:794: delete oid 48 current snap is 93 2023-06-27T12:41:41.497 INFO:tasks.rados.rados.0.smithi138.stdout:794: done (4 left) 2023-06-27T12:41:41.497 INFO:tasks.rados.rados.0.smithi138.stdout:795: snap_remove snap 92 2023-06-27T12:41:42.444 INFO:tasks.rados.rados.0.smithi138.stdout:795: done (4 left) 2023-06-27T12:41:42.444 INFO:tasks.rados.rados.0.smithi138.stdout:796: read oid 21 snap -1 2023-06-27T12:41:42.444 INFO:tasks.rados.rados.0.smithi138.stdout:796: expect (ObjNum 265 snap 70 seq_num 265) 2023-06-27T12:41:42.444 INFO:tasks.rados.rados.0.smithi138.stdout:797: write oid 39 current snap is 93 2023-06-27T12:41:42.445 INFO:tasks.rados.rados.0.smithi138.stdout:797: seq_num 383 ranges {437081=638017,1679074=582789} 2023-06-27T12:41:42.450 INFO:tasks.rados.rados.0.smithi138.stdout:797: writing smithi138138488-39 from 437081 to 1075098 tid 1 2023-06-27T12:41:42.459 INFO:tasks.rados.rados.0.smithi138.stdout:797: writing smithi138138488-39 from 1679074 to 2261863 tid 2 2023-06-27T12:41:42.459 INFO:tasks.rados.rados.0.smithi138.stdout:798: write (excl) oid 44 current snap is 93 2023-06-27T12:41:42.459 INFO:tasks.rados.rados.0.smithi138.stdout:798: seq_num 384 ranges {453888=713266,1737556=442800} 2023-06-27T12:41:42.468 INFO:tasks.rados.rados.0.smithi138.stdout:798: writing smithi138138488-44 from 453888 to 1167154 tid 1 2023-06-27T12:41:42.474 INFO:tasks.rados.rados.0.smithi138.stdout:798: writing smithi138138488-44 from 1737556 to 2180356 tid 2 2023-06-27T12:41:42.474 INFO:tasks.rados.rados.0.smithi138.stdout:799: delete oid 6 current snap is 93 2023-06-27T12:41:42.474 INFO:tasks.rados.rados.0.smithi138.stdout:792: expect (ObjNum 364 snap 91 seq_num 364) 2023-06-27T12:41:42.562 INFO:tasks.rados.rados.0.smithi138.stdout:791: expect (ObjNum 331 snap 83 seq_num 331) 2023-06-27T12:41:42.788 INFO:tasks.rados.rados.0.smithi138.stdout:790: finishing write tid 1 to smithi138138488-9 2023-06-27T12:41:42.788 INFO:tasks.rados.rados.0.smithi138.stdout:790: finishing write tid 2 to smithi138138488-9 2023-06-27T12:41:42.788 INFO:tasks.rados.rados.0.smithi138.stdout:790: finishing write tid 3 to smithi138138488-9 2023-06-27T12:41:42.789 INFO:tasks.rados.rados.0.smithi138.stdout:793: finishing write tid 1 to smithi138138488-29 2023-06-27T12:41:42.789 INFO:tasks.rados.rados.0.smithi138.stdout:790: finishing write tid 4 to smithi138138488-9 2023-06-27T12:41:42.789 INFO:tasks.rados.rados.0.smithi138.stdout:790: oid 9 updating version 0 to 242 2023-06-27T12:41:42.789 INFO:tasks.rados.rados.0.smithi138.stdout:790: oid 9 version 242 is already newer than 242 2023-06-27T12:41:42.789 INFO:tasks.rados.rados.0.smithi138.stdout:790: oid 9 version 242 is already newer than 242 2023-06-27T12:41:42.789 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 242 (ObjNum 380 snap 93 seq_num 380) dirty exists 2023-06-27T12:41:42.789 INFO:tasks.rados.rados.0.smithi138.stdout:790: left oid 9 (ObjNum 380 snap 93 seq_num 380) 2023-06-27T12:41:42.790 INFO:tasks.rados.rados.0.smithi138.stdout:793: finishing write tid 2 to smithi138138488-29 2023-06-27T12:41:42.790 INFO:tasks.rados.rados.0.smithi138.stdout:793: finishing write tid 3 to smithi138138488-29 2023-06-27T12:41:42.790 INFO:tasks.rados.rados.0.smithi138.stdout:793: finishing write tid 4 to smithi138138488-29 2023-06-27T12:41:42.790 INFO:tasks.rados.rados.0.smithi138.stdout:793: oid 29 updating version 0 to 81 2023-06-27T12:41:42.790 INFO:tasks.rados.rados.0.smithi138.stdout:793: oid 29 version 81 is already newer than 81 2023-06-27T12:41:42.790 INFO:tasks.rados.rados.0.smithi138.stdout:793: oid 29 version 81 is already newer than 81 2023-06-27T12:41:42.790 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 81 (ObjNum 381 snap 93 seq_num 381) dirty exists 2023-06-27T12:41:42.790 INFO:tasks.rados.rados.0.smithi138.stdout:793: left oid 29 (ObjNum 381 snap 93 seq_num 381) 2023-06-27T12:41:42.791 INFO:tasks.rados.rados.0.smithi138.stdout:790: done (7 left) 2023-06-27T12:41:42.791 INFO:tasks.rados.rados.0.smithi138.stdout:791: done (6 left) 2023-06-27T12:41:42.791 INFO:tasks.rados.rados.0.smithi138.stdout:792: done (5 left) 2023-06-27T12:41:42.791 INFO:tasks.rados.rados.0.smithi138.stdout:793: done (4 left) 2023-06-27T12:41:42.791 INFO:tasks.rados.rados.0.smithi138.stdout:799: done (3 left) 2023-06-27T12:41:42.791 INFO:tasks.rados.rados.0.smithi138.stdout:800: read oid 34 snap -1 2023-06-27T12:41:42.791 INFO:tasks.rados.rados.0.smithi138.stdout:800: expect (ObjNum 342 snap 84 seq_num 342) 2023-06-27T12:41:42.791 INFO:tasks.rados.rados.0.smithi138.stdout:801: read oid 16 snap -1 2023-06-27T12:41:42.792 INFO:tasks.rados.rados.0.smithi138.stdout:801: expect (ObjNum 363 snap 91 seq_num 363) 2023-06-27T12:41:42.792 INFO:tasks.rados.rados.0.smithi138.stdout:802: snap_create 2023-06-27T12:41:42.792 INFO:tasks.rados.rados.0.smithi138.stdout:796: expect (ObjNum 265 snap 70 seq_num 265) 2023-06-27T12:41:42.918 INFO:tasks.rados.rados.0.smithi138.stdout:797: finishing write tid 1 to smithi138138488-39 2023-06-27T12:41:42.918 INFO:tasks.rados.rados.0.smithi138.stdout:797: finishing write tid 2 to smithi138138488-39 2023-06-27T12:41:42.918 INFO:tasks.rados.rados.0.smithi138.stdout:798: finishing write tid 1 to smithi138138488-44 2023-06-27T12:41:42.919 INFO:tasks.rados.rados.0.smithi138.stdout:797: finishing write tid 3 to smithi138138488-39 2023-06-27T12:41:42.919 INFO:tasks.rados.rados.0.smithi138.stdout:797: finishing write tid 4 to smithi138138488-39 2023-06-27T12:41:42.919 INFO:tasks.rados.rados.0.smithi138.stdout:797: oid 39 updating version 0 to 132 2023-06-27T12:41:42.919 INFO:tasks.rados.rados.0.smithi138.stdout:797: oid 39 version 132 is already newer than 132 2023-06-27T12:41:42.919 INFO:tasks.rados.rados.0.smithi138.stdout:797: oid 39 version 132 is already newer than 132 2023-06-27T12:41:42.919 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 39 v 132 (ObjNum 383 snap 93 seq_num 383) dirty exists 2023-06-27T12:41:42.919 INFO:tasks.rados.rados.0.smithi138.stdout:797: left oid 39 (ObjNum 383 snap 93 seq_num 383) 2023-06-27T12:41:42.919 INFO:tasks.rados.rados.0.smithi138.stdout:798: finishing write tid 2 to smithi138138488-44 2023-06-27T12:41:42.920 INFO:tasks.rados.rados.0.smithi138.stdout:798: finishing write tid 3 to smithi138138488-44 2023-06-27T12:41:42.920 INFO:tasks.rados.rados.0.smithi138.stdout:798: finishing write tid 4 to smithi138138488-44 2023-06-27T12:41:42.920 INFO:tasks.rados.rados.0.smithi138.stdout:798: oid 44 updating version 0 to 270 2023-06-27T12:41:42.920 INFO:tasks.rados.rados.0.smithi138.stdout:798: oid 44 version 270 is already newer than 268 2023-06-27T12:41:42.920 INFO:tasks.rados.rados.0.smithi138.stdout:798: oid 44 version 270 is already newer than 270 2023-06-27T12:41:42.920 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 270 (ObjNum 384 snap 93 seq_num 384) dirty exists 2023-06-27T12:41:42.920 INFO:tasks.rados.rados.0.smithi138.stdout:798: left oid 44 (ObjNum 384 snap 93 seq_num 384) 2023-06-27T12:41:42.921 INFO:tasks.rados.rados.0.smithi138.stdout:800: expect (ObjNum 342 snap 84 seq_num 342) 2023-06-27T12:41:43.058 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:43.150 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:43.151 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:43.151 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:43.155 INFO:tasks.rados.rados.0.smithi138.stdout:801: expect (ObjNum 363 snap 91 seq_num 363) 2023-06-27T12:41:43.190 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:43.185+0000 7fc87de59700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:43.193 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:43.189+0000 7fc87de59700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:43.247 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:43.250 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:43.250 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:43.285 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:43.278+0000 7f9e4f009700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:43.288 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:43.284+0000 7f9e4f009700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:43.449 INFO:tasks.rados.rados.0.smithi138.stdout:796: done (5 left) 2023-06-27T12:41:43.450 INFO:tasks.rados.rados.0.smithi138.stdout:797: done (4 left) 2023-06-27T12:41:43.450 INFO:tasks.rados.rados.0.smithi138.stdout:798: done (3 left) 2023-06-27T12:41:43.450 INFO:tasks.rados.rados.0.smithi138.stdout:800: done (2 left) 2023-06-27T12:41:43.450 INFO:tasks.rados.rados.0.smithi138.stdout:801: done (1 left) 2023-06-27T12:41:43.450 INFO:tasks.rados.rados.0.smithi138.stdout:802: done (0 left) 2023-06-27T12:41:43.450 INFO:tasks.rados.rados.0.smithi138.stdout:803: snap_remove snap 93 2023-06-27T12:41:43.497 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:43.500 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:43.511 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2023-06-27T12:41:43.444609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":79834702,"num_objects":198,"num_object_clones":156,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":1107,"num_read_kb":774877,"num_write":977,"num_write_kb":283174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2048,"ondisk_log_size":2048,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":758920,"kb_used_data":622400,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376728440,"statfs":{"total":386547056640,"available":385769922560,"internally_reserved":0,"allocated":637337600,"data_stored":729555967,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023150"},"pg_stats":[{"pgid":"3.f","version":"205'10","reported_seq":477,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.756870+0000","last_change":"2023-06-27T12:41:38.744951+0000","last_active":"2023-06-27T12:41:38.756870+0000","last_peered":"2023-06-27T12:41:38.756870+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:38.756870+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2235397,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":8,"num_write_kb":2199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"205'115","reported_seq":641,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.620145+0000","last_change":"2023-06-27T12:41:38.570405+0000","last_active":"2023-06-27T12:41:38.620145+0000","last_peered":"2023-06-27T12:41:38.620145+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:38.620145+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5613637,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":88,"num_read_kb":102752,"num_write":77,"num_write_kb":22764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"198'151","reported_seq":633,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.464878+0000","last_change":"2023-06-27T12:41:38.464878+0000","last_active":"2023-06-27T12:41:38.464878+0000","last_peered":"2023-06-27T12:41:38.464878+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:38.464878+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":19,"num_object_clones":14,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":79,"num_read_kb":31660,"num_write":107,"num_write_kb":31933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"184'74","reported_seq":589,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.519054+0000","last_change":"2023-06-27T12:41:38.537360+0000","last_active":"2023-06-27T12:41:38.519054+0000","last_peered":"2023-06-27T12:41:38.519054+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:38.519054+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563117,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":66,"num_read_kb":31562,"num_write":56,"num_write_kb":15640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"205'42","reported_seq":166,"reported_epoch":205,"state":"active+undersized+degraded+snaptrim","last_fresh":"2023-06-27T12:41:38.676630+0000","last_change":"2023-06-27T12:41:38.479467+0000","last_active":"2023-06-27T12:41:38.676630+0000","last_peered":"2023-06-27T12:41:38.676630+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:38.676630+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":1,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":24673,"num_write":9,"num_write_kb":2549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"205'264","reported_seq":626,"reported_epoch":205,"state":"active+clean+snaptrim","last_fresh":"2023-06-27T12:41:38.618743+0000","last_change":"2023-06-27T12:41:38.476772+0000","last_active":"2023-06-27T12:41:38.618743+0000","last_peered":"2023-06-27T12:41:38.618743+0000","last_clean":"2023-06-27T12:41:38.618743+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:38.618743+0000","last_undegraded":"2023-06-27T12:41:38.618743+0000","last_fullsized":"2023-06-27T12:41:38.618743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9966782,"num_objects":31,"num_object_clones":25,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":184,"num_read_kb":118398,"num_write":145,"num_write_kb":41291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"205'142","reported_seq":519,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-06-27T12:41:38.853742+0000","last_change":"2023-06-27T12:41:38.550009+0000","last_active":"2023-06-27T12:41:38.853742+0000","last_peered":"2023-06-27T12:41:38.853742+0000","last_clean":"2023-06-27T12:41:38.853742+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:38.853742+0000","last_undegraded":"2023-06-27T12:41:38.853742+0000","last_fullsized":"2023-06-27T12:41:38.853742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7916262,"num_objects":17,"num_object_clones":14,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":44590,"num_write":80,"num_write_kb":23093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"199'60","reported_seq":550,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.455158+0000","last_change":"2023-06-27T12:41:38.461411+0000","last_active":"2023-06-27T12:41:38.455158+0000","last_peered":"2023-06-27T12:41:38.455158+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:38.455158+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916064,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":332,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-06-27T12:41:38.558898+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:38.558898+0000","last_peered":"2023-06-27T12:41:38.558898+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:38.558898+0000","last_undegraded":"2023-06-27T12:41:38.558898+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":472,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-06-27T12:41:38.438599+0000","last_change":"2023-06-27T12:41:38.464090+0000","last_active":"2023-06-27T12:41:38.438599+0000","last_peered":"2023-06-27T12:41:38.438599+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:38.438599+0000","last_undegraded":"2023-06-27T12:41:38.438599+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":309,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-06-27T12:41:38.475368+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:38.475368+0000","last_peered":"2023-06-27T12:41:38.475368+0000","last_clean":"2023-06-27T12:41:38.475368+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:38.475368+0000","last_undegraded":"2023-06-27T12:41:38.475368+0000","last_fullsized":"2023-06-27T12:41:38.475368+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"205'107","reported_seq":600,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.966607+0000","last_change":"2023-06-27T12:41:38.573162+0000","last_active":"2023-06-27T12:41:38.966607+0000","last_peered":"2023-06-27T12:41:38.966607+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:38.966607+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5143041,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":54,"num_read_kb":56875,"num_write":68,"num_write_kb":18284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":13}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":316,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-06-27T12:41:38.447088+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:38.447088+0000","last_peered":"2023-06-27T12:41:38.447088+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:38.447088+0000","last_undegraded":"2023-06-27T12:41:38.447088+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"205'299","reported_seq":234,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.979594+0000","last_change":"2023-06-27T12:41:38.776933+0000","last_active":"2023-06-27T12:41:38.979594+0000","last_peered":"2023-06-27T12:41:38.979594+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:38.979594+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1210203,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":23,"num_read_kb":12412,"num_write":42,"num_write_kb":12514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":313,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-06-27T12:41:38.440207+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:38.440207+0000","last_peered":"2023-06-27T12:41:38.440207+0000","last_clean":"2023-06-27T12:41:38.440207+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:38.440207+0000","last_undegraded":"2023-06-27T12:41:38.440207+0000","last_fullsized":"2023-06-27T12:41:38.440207+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":61,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-06-27T12:41:38.503916+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:38.503916+0000","last_peered":"2023-06-27T12:41:38.503916+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:38.503916+0000","last_undegraded":"2023-06-27T12:41:38.503916+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"205'232","reported_seq":736,"reported_epoch":205,"state":"active+undersized+degraded+snaptrim","last_fresh":"2023-06-27T12:41:38.706348+0000","last_change":"2023-06-27T12:41:38.447023+0000","last_active":"2023-06-27T12:41:38.706348+0000","last_peered":"2023-06-27T12:41:38.706348+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:38.706348+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":10,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14168499,"num_objects":32,"num_object_clones":26,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":198,"num_read_kb":119568,"num_write":170,"num_write_kb":48856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":309,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-06-27T12:41:38.478775+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:38.478775+0000","last_peered":"2023-06-27T12:41:38.478775+0000","last_clean":"2023-06-27T12:41:38.478775+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:38.478775+0000","last_undegraded":"2023-06-27T12:41:38.478775+0000","last_fullsized":"2023-06-27T12:41:38.478775+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"205'129","reported_seq":196,"reported_epoch":205,"state":"active+undersized+degraded+snaptrim","last_fresh":"2023-06-27T12:41:38.972765+0000","last_change":"2023-06-27T12:41:37.464551+0000","last_active":"2023-06-27T12:41:38.972765+0000","last_peered":"2023-06-27T12:41:38.972765+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:38.972765+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":55,"num_read_kb":35681,"num_write":21,"num_write_kb":6190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":75,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-06-27T12:41:38.555087+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:38.555087+0000","last_peered":"2023-06-27T12:41:38.555087+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:38.555087+0000","last_undegraded":"2023-06-27T12:41:38.555087+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"204'122","reported_seq":627,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.562964+0000","last_change":"2023-06-27T12:41:38.563524+0000","last_active":"2023-06-27T12:41:38.562964+0000","last_peered":"2023-06-27T12:41:38.562964+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:38.562964+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8353211,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":71,"num_read_kb":71931,"num_write":86,"num_write_kb":25844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"201'77","reported_seq":190,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.609495+0000","last_change":"2023-06-27T12:41:38.455560+0000","last_active":"2023-06-27T12:41:38.609495+0000","last_peered":"2023-06-27T12:41:38.609495+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:38.609495+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3983292,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":38,"num_read_kb":21760,"num_write":14,"num_write_kb":4234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":332,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-06-27T12:41:38.556473+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:38.556473+0000","last_peered":"2023-06-27T12:41:38.556473+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:38.556473+0000","last_undegraded":"2023-06-27T12:41:38.556473+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":313,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-06-27T12:41:38.457584+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:38.457584+0000","last_peered":"2023-06-27T12:41:38.457584+0000","last_clean":"2023-06-27T12:41:38.457584+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:38.457584+0000","last_undegraded":"2023-06-27T12:41:38.457584+0000","last_fullsized":"2023-06-27T12:41:38.457584+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"201'127","reported_seq":604,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:38.589882+0000","last_change":"2023-06-27T12:41:38.588885+0000","last_active":"2023-06-27T12:41:38.589882+0000","last_peered":"2023-06-27T12:41:38.589882+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:38.589882+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117715,"num_objects":13,"num_object_clones":10,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":62,"num_read_kb":60134,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":13}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":79834702,"num_objects":198,"num_object_clones":156,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":1107,"num_read_kb":774877,"num_write":977,"num_write_kb":283174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1951,"ondisk_log_size":1951,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":176276,"kb_used_data":140092,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94195564,"statfs":{"total":96636764160,"available":96456257536,"internally_reserved":0,"allocated":143454208,"data_stored":163492906,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":193284,"kb_used_data":158760,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94178556,"statfs":{"total":96636764160,"available":96438841344,"internally_reserved":0,"allocated":162570240,"data_stored":185692653,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":208080,"kb_used_data":173188,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94163760,"statfs":{"total":96636764160,"available":96423690240,"internally_reserved":0,"allocated":177344512,"data_stored":203330856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:44.454 INFO:tasks.rados.rados.0.smithi138.stdout:803: done (0 left) 2023-06-27T12:41:44.455 INFO:tasks.rados.rados.0.smithi138.stdout:804: read oid 29 snap -1 2023-06-27T12:41:44.455 INFO:tasks.rados.rados.0.smithi138.stdout:804: expect (ObjNum 381 snap 93 seq_num 381) 2023-06-27T12:41:44.455 INFO:tasks.rados.rados.0.smithi138.stdout:805: read oid 3 snap -1 2023-06-27T12:41:44.455 INFO:tasks.rados.rados.0.smithi138.stdout:805: expect deleted 2023-06-27T12:41:44.455 INFO:tasks.rados.rados.0.smithi138.stdout:806: snap_create 2023-06-27T12:41:44.523 INFO:tasks.rados.rados.0.smithi138.stdout:804: expect (ObjNum 381 snap 93 seq_num 381) 2023-06-27T12:41:45.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:45.384 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:45.379+0000 7fcaef3b4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:45.387 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:45.383+0000 7fcaef3b4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:45.419 INFO:tasks.rados.rados.0.smithi138.stdout:804: done (2 left) 2023-06-27T12:41:45.420 INFO:tasks.rados.rados.0.smithi138.stdout:805: done (1 left) 2023-06-27T12:41:45.420 INFO:tasks.rados.rados.0.smithi138.stdout:806: done (0 left) 2023-06-27T12:41:45.420 INFO:tasks.rados.rados.0.smithi138.stdout:807: delete oid 11 current snap is 95 2023-06-27T12:41:45.434 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:45.437 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:45.438 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:45.455 INFO:tasks.rados.rados.0.smithi138.stdout:807: done (0 left) 2023-06-27T12:41:45.456 INFO:tasks.rados.rados.0.smithi138.stdout:808: snap_remove snap 94 2023-06-27T12:41:46.457 INFO:tasks.rados.rados.0.smithi138.stdout:808: done (0 left) 2023-06-27T12:41:46.457 INFO:tasks.rados.rados.0.smithi138.stdout:809: read oid 44 snap -1 2023-06-27T12:41:46.457 INFO:tasks.rados.rados.0.smithi138.stdout:809: expect (ObjNum 384 snap 93 seq_num 384) 2023-06-27T12:41:46.457 INFO:tasks.rados.rados.0.smithi138.stdout:810: write oid 16 current snap is 95 2023-06-27T12:41:46.457 INFO:tasks.rados.rados.0.smithi138.stdout:810: seq_num 387 ranges {504309=539013,1796649=590127} 2023-06-27T12:41:46.463 INFO:tasks.rados.rados.0.smithi138.stdout:810: writing smithi138138488-16 from 504309 to 1043322 tid 1 2023-06-27T12:41:46.473 INFO:tasks.rados.rados.0.smithi138.stdout:810: writing smithi138138488-16 from 1796649 to 2386776 tid 2 2023-06-27T12:41:46.473 INFO:tasks.rados.rados.0.smithi138.stdout:811: read oid 24 snap -1 2023-06-27T12:41:46.474 INFO:tasks.rados.rados.0.smithi138.stdout:811: expect (ObjNum 379 snap 91 seq_num 379) 2023-06-27T12:41:46.474 INFO:tasks.rados.rados.0.smithi138.stdout:812: write oid 19 current snap is 95 2023-06-27T12:41:46.474 INFO:tasks.rados.rados.0.smithi138.stdout:812: seq_num 388 ranges {521116=730615,1971484=549632} 2023-06-27T12:41:46.482 INFO:tasks.rados.rados.0.smithi138.stdout:812: writing smithi138138488-19 from 521116 to 1251731 tid 1 2023-06-27T12:41:46.491 INFO:tasks.rados.rados.0.smithi138.stdout:812: writing smithi138138488-19 from 1971484 to 2521116 tid 2 2023-06-27T12:41:46.491 INFO:tasks.rados.rados.0.smithi138.stdout:813: read oid 42 snap -1 2023-06-27T12:41:46.491 INFO:tasks.rados.rados.0.smithi138.stdout:813: expect deleted 2023-06-27T12:41:46.491 INFO:tasks.rados.rados.0.smithi138.stdout:814: write (excl) oid 11 current snap is 95 2023-06-27T12:41:46.491 INFO:tasks.rados.rados.0.smithi138.stdout:814: seq_num 389 ranges {537923=405864,1513613=593796} 2023-06-27T12:41:46.497 INFO:tasks.rados.rados.0.smithi138.stdout:814: writing smithi138138488-11 from 537923 to 943787 tid 1 2023-06-27T12:41:46.506 INFO:tasks.rados.rados.0.smithi138.stdout:814: writing smithi138138488-11 from 1513613 to 2107409 tid 2 2023-06-27T12:41:46.506 INFO:tasks.rados.rados.0.smithi138.stdout:815: delete oid 13 current snap is 95 2023-06-27T12:41:46.542 INFO:tasks.rados.rados.0.smithi138.stdout:809: expect (ObjNum 384 snap 93 seq_num 384) 2023-06-27T12:41:46.722 INFO:tasks.rados.rados.0.smithi138.stdout:810: finishing write tid 1 to smithi138138488-16 2023-06-27T12:41:46.722 INFO:tasks.rados.rados.0.smithi138.stdout:810: finishing write tid 2 to smithi138138488-16 2023-06-27T12:41:46.722 INFO:tasks.rados.rados.0.smithi138.stdout:812: finishing write tid 1 to smithi138138488-19 2023-06-27T12:41:46.723 INFO:tasks.rados.rados.0.smithi138.stdout:810: finishing write tid 3 to smithi138138488-16 2023-06-27T12:41:46.723 INFO:tasks.rados.rados.0.smithi138.stdout:810: finishing write tid 4 to smithi138138488-16 2023-06-27T12:41:46.723 INFO:tasks.rados.rados.0.smithi138.stdout:810: oid 16 updating version 0 to 273 2023-06-27T12:41:46.723 INFO:tasks.rados.rados.0.smithi138.stdout:810: oid 16 version 273 is already newer than 273 2023-06-27T12:41:46.723 INFO:tasks.rados.rados.0.smithi138.stdout:810: oid 16 version 273 is already newer than 273 2023-06-27T12:41:46.723 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 273 (ObjNum 387 snap 95 seq_num 387) dirty exists 2023-06-27T12:41:46.723 INFO:tasks.rados.rados.0.smithi138.stdout:810: left oid 16 (ObjNum 387 snap 95 seq_num 387) 2023-06-27T12:41:46.723 INFO:tasks.rados.rados.0.smithi138.stdout:812: finishing write tid 2 to smithi138138488-19 2023-06-27T12:41:46.724 INFO:tasks.rados.rados.0.smithi138.stdout:811: expect (ObjNum 379 snap 91 seq_num 379) 2023-06-27T12:41:46.810 INFO:tasks.rados.rados.0.smithi138.stdout:812: finishing write tid 3 to smithi138138488-19 2023-06-27T12:41:46.810 INFO:tasks.rados.rados.0.smithi138.stdout:812: finishing write tid 4 to smithi138138488-19 2023-06-27T12:41:46.810 INFO:tasks.rados.rados.0.smithi138.stdout:812: oid 19 updating version 0 to 247 2023-06-27T12:41:46.811 INFO:tasks.rados.rados.0.smithi138.stdout:812: oid 19 version 247 is already newer than 247 2023-06-27T12:41:46.811 INFO:tasks.rados.rados.0.smithi138.stdout:812: oid 19 version 247 is already newer than 247 2023-06-27T12:41:46.811 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 247 (ObjNum 388 snap 95 seq_num 388) dirty exists 2023-06-27T12:41:46.811 INFO:tasks.rados.rados.0.smithi138.stdout:812: left oid 19 (ObjNum 388 snap 95 seq_num 388) 2023-06-27T12:41:46.811 INFO:tasks.rados.rados.0.smithi138.stdout:809: done (6 left) 2023-06-27T12:41:46.811 INFO:tasks.rados.rados.0.smithi138.stdout:810: done (5 left) 2023-06-27T12:41:46.811 INFO:tasks.rados.rados.0.smithi138.stdout:811: done (4 left) 2023-06-27T12:41:46.812 INFO:tasks.rados.rados.0.smithi138.stdout:812: done (3 left) 2023-06-27T12:41:46.812 INFO:tasks.rados.rados.0.smithi138.stdout:813: done (2 left) 2023-06-27T12:41:46.812 INFO:tasks.rados.rados.0.smithi138.stdout:815: done (1 left) 2023-06-27T12:41:46.812 INFO:tasks.rados.rados.0.smithi138.stdout:816: delete oid 2 current snap is 95 2023-06-27T12:41:46.812 INFO:tasks.rados.rados.0.smithi138.stdout:814: finishing write tid 1 to smithi138138488-11 2023-06-27T12:41:46.812 INFO:tasks.rados.rados.0.smithi138.stdout:814: finishing write tid 2 to smithi138138488-11 2023-06-27T12:41:46.813 INFO:tasks.rados.rados.0.smithi138.stdout:814: finishing write tid 3 to smithi138138488-11 2023-06-27T12:41:46.813 INFO:tasks.rados.rados.0.smithi138.stdout:814: finishing write tid 4 to smithi138138488-11 2023-06-27T12:41:46.813 INFO:tasks.rados.rados.0.smithi138.stdout:814: oid 11 updating version 0 to 159 2023-06-27T12:41:46.813 INFO:tasks.rados.rados.0.smithi138.stdout:814: oid 11 updating version 159 to 161 2023-06-27T12:41:46.813 INFO:tasks.rados.rados.0.smithi138.stdout:814: oid 11 version 161 is already newer than 161 2023-06-27T12:41:46.813 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 11 v 161 (ObjNum 389 snap 95 seq_num 389) dirty exists 2023-06-27T12:41:46.813 INFO:tasks.rados.rados.0.smithi138.stdout:814: left oid 11 (ObjNum 389 snap 95 seq_num 389) 2023-06-27T12:41:46.855 INFO:tasks.rados.rados.0.smithi138.stdout:814: done (1 left) 2023-06-27T12:41:46.856 INFO:tasks.rados.rados.0.smithi138.stdout:816: done (0 left) 2023-06-27T12:41:46.856 INFO:tasks.rados.rados.0.smithi138.stdout:817: read oid 45 snap -1 2023-06-27T12:41:46.856 INFO:tasks.rados.rados.0.smithi138.stdout:817: expect (ObjNum 322 snap 81 seq_num 322) 2023-06-27T12:41:46.856 INFO:tasks.rados.rados.0.smithi138.stdout:818: delete oid 25 current snap is 95 2023-06-27T12:41:46.859 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~72s 2023-06-27T12:41:46.873 INFO:tasks.rados.rados.0.smithi138.stdout:817: expect (ObjNum 322 snap 81 seq_num 322) 2023-06-27T12:41:47.089 INFO:tasks.rados.rados.0.smithi138.stdout:817: done (1 left) 2023-06-27T12:41:47.089 INFO:tasks.rados.rados.0.smithi138.stdout:818: done (0 left) 2023-06-27T12:41:47.089 INFO:tasks.rados.rados.0.smithi138.stdout:819: read oid 26 snap -1 2023-06-27T12:41:47.089 INFO:tasks.rados.rados.0.smithi138.stdout:819: expect (ObjNum 360 snap 91 seq_num 360) 2023-06-27T12:41:47.089 INFO:tasks.rados.rados.0.smithi138.stdout:820: write (excl) oid 16 current snap is 95 2023-06-27T12:41:47.089 INFO:tasks.rados.rados.0.smithi138.stdout:820: seq_num 393 ranges {605151=706860,2031188=573963} 2023-06-27T12:41:47.096 INFO:tasks.rados.rados.0.smithi138.stdout:820: writing smithi138138488-16 from 605151 to 1312011 tid 1 2023-06-27T12:41:47.104 INFO:tasks.rados.rados.0.smithi138.stdout:820: writing smithi138138488-16 from 2031188 to 2605151 tid 2 2023-06-27T12:41:47.105 INFO:tasks.rados.rados.0.smithi138.stdout:821: snap_create 2023-06-27T12:41:47.105 INFO:tasks.rados.rados.0.smithi138.stdout:819: expect (ObjNum 360 snap 91 seq_num 360) 2023-06-27T12:41:47.213 INFO:tasks.rados.rados.0.smithi138.stdout:820: finishing write tid 1 to smithi138138488-16 2023-06-27T12:41:47.213 INFO:tasks.rados.rados.0.smithi138.stdout:820: finishing write tid 2 to smithi138138488-16 2023-06-27T12:41:47.213 INFO:tasks.rados.rados.0.smithi138.stdout:820: finishing write tid 3 to smithi138138488-16 2023-06-27T12:41:47.214 INFO:tasks.rados.rados.0.smithi138.stdout:820: finishing write tid 4 to smithi138138488-16 2023-06-27T12:41:47.214 INFO:tasks.rados.rados.0.smithi138.stdout:820: oid 16 updating version 0 to 274 2023-06-27T12:41:47.214 INFO:tasks.rados.rados.0.smithi138.stdout:820: oid 16 updating version 274 to 276 2023-06-27T12:41:47.214 INFO:tasks.rados.rados.0.smithi138.stdout:820: oid 16 version 276 is already newer than 276 2023-06-27T12:41:47.214 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 276 (ObjNum 393 snap 95 seq_num 393) dirty exists 2023-06-27T12:41:47.214 INFO:tasks.rados.rados.0.smithi138.stdout:820: left oid 16 (ObjNum 393 snap 95 seq_num 393) 2023-06-27T12:41:47.439 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:47.460 INFO:tasks.rados.rados.0.smithi138.stdout:819: done (2 left) 2023-06-27T12:41:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:820: done (1 left) 2023-06-27T12:41:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:821: done (0 left) 2023-06-27T12:41:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:822: delete oid 23 current snap is 96 2023-06-27T12:41:47.493 INFO:tasks.rados.rados.0.smithi138.stdout:822: done (0 left) 2023-06-27T12:41:47.494 INFO:tasks.rados.rados.0.smithi138.stdout:823: delete oid 27 current snap is 96 2023-06-27T12:41:47.541 INFO:tasks.rados.rados.0.smithi138.stdout:823: done (0 left) 2023-06-27T12:41:47.541 INFO:tasks.rados.rados.0.smithi138.stdout:824: delete oid 18 current snap is 96 2023-06-27T12:41:47.577 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:47.571+0000 7f7ba4eea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:47.580 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:47.576+0000 7f7ba4eea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:47.589 INFO:tasks.rados.rados.0.smithi138.stdout:824: done (0 left) 2023-06-27T12:41:47.590 INFO:tasks.rados.rados.0.smithi138.stdout:825: delete oid 39 current snap is 96 2023-06-27T12:41:47.632 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:47.635 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:47.636 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:47.662 INFO:tasks.rados.rados.0.smithi138.stdout:825: done (0 left) 2023-06-27T12:41:47.662 INFO:tasks.rados.rados.0.smithi138.stdout:826: snap_remove snap 95 2023-06-27T12:41:48.465 INFO:tasks.rados.rados.0.smithi138.stdout:826: done (0 left) 2023-06-27T12:41:48.465 INFO:tasks.rados.rados.0.smithi138.stdout:827: read oid 6 snap -1 2023-06-27T12:41:48.465 INFO:tasks.rados.rados.0.smithi138.stdout:827: expect deleted 2023-06-27T12:41:48.465 INFO:tasks.rados.rados.0.smithi138.stdout:828: delete oid 47 current snap is 96 2023-06-27T12:41:48.503 INFO:tasks.rados.rados.0.smithi138.stdout:828: done (1 left) 2023-06-27T12:41:48.503 INFO:tasks.rados.rados.0.smithi138.stdout:829: write oid 25 current snap is 96 2023-06-27T12:41:48.503 INFO:tasks.rados.rados.0.smithi138.stdout:829: seq_num 399 ranges {705993=474707,1582080=612141} 2023-06-27T12:41:48.511 INFO:tasks.rados.rados.0.smithi138.stdout:829: writing smithi138138488-25 from 705993 to 1180700 tid 1 2023-06-27T12:41:48.513 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:48.513 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:48.514 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:48.518 INFO:tasks.rados.rados.0.smithi138.stdout:829: writing smithi138138488-25 from 1582080 to 2194221 tid 2 2023-06-27T12:41:48.518 INFO:tasks.rados.rados.0.smithi138.stdout:830: snap_create 2023-06-27T12:41:48.607 INFO:tasks.rados.rados.0.smithi138.stdout:829: finishing write tid 1 to smithi138138488-25 2023-06-27T12:41:48.611 INFO:tasks.rados.rados.0.smithi138.stdout:829: finishing write tid 2 to smithi138138488-25 2023-06-27T12:41:48.633 INFO:tasks.rados.rados.0.smithi138.stdout:829: finishing write tid 3 to smithi138138488-25 2023-06-27T12:41:48.634 INFO:tasks.rados.rados.0.smithi138.stdout:829: finishing write tid 4 to smithi138138488-25 2023-06-27T12:41:48.634 INFO:tasks.rados.rados.0.smithi138.stdout:829: oid 25 updating version 0 to 85 2023-06-27T12:41:48.634 INFO:tasks.rados.rados.0.smithi138.stdout:829: oid 25 version 85 is already newer than 85 2023-06-27T12:41:48.635 INFO:tasks.rados.rados.0.smithi138.stdout:829: oid 25 version 85 is already newer than 85 2023-06-27T12:41:48.635 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 85 (ObjNum 399 snap 96 seq_num 399) dirty exists 2023-06-27T12:41:48.635 INFO:tasks.rados.rados.0.smithi138.stdout:829: left oid 25 (ObjNum 399 snap 96 seq_num 399) 2023-06-27T12:41:48.656 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:48.649+0000 7ff551c49700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:48.659 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:48.656+0000 7ff551c49700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:48.856 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:48.858 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:48.869 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2023-06-27T12:41:48.460149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":79683505,"num_objects":199,"num_object_clones":157,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":1166,"num_read_kb":821588,"num_write":997,"num_write_kb":288767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2078,"ondisk_log_size":2078,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":737360,"kb_used_data":610804,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376750000,"statfs":{"total":386547056640,"available":385792000000,"internally_reserved":0,"allocated":625463296,"data_stored":712878677,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.011971"},"pg_stats":[{"pgid":"3.f","version":"205'10","reported_seq":487,"reported_epoch":209,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:42.483468+0000","last_change":"2023-06-27T12:41:42.490949+0000","last_active":"2023-06-27T12:41:42.483468+0000","last_peered":"2023-06-27T12:41:42.483468+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:42.483468+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2235397,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":8,"num_write_kb":2199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"205'115","reported_seq":651,"reported_epoch":209,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:42.457685+0000","last_change":"2023-06-27T12:41:42.464093+0000","last_active":"2023-06-27T12:41:42.457685+0000","last_peered":"2023-06-27T12:41:42.457685+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:42.457685+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5613637,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":88,"num_read_kb":102752,"num_write":77,"num_write_kb":22764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"205'154","reported_seq":648,"reported_epoch":210,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:43.462240+0000","last_change":"2023-06-27T12:41:42.490540+0000","last_active":"2023-06-27T12:41:43.462240+0000","last_peered":"2023-06-27T12:41:43.462240+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:43.462240+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210302,"num_objects":19,"num_object_clones":14,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":86,"num_read_kb":37782,"num_write":107,"num_write_kb":31933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"184'74","reported_seq":602,"reported_epoch":209,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:42.489285+0000","last_change":"2023-06-27T12:41:42.507886+0000","last_active":"2023-06-27T12:41:42.489285+0000","last_peered":"2023-06-27T12:41:42.489285+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:42.489285+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563117,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":73,"num_read_kb":40226,"num_write":56,"num_write_kb":15640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"205'42","reported_seq":179,"reported_epoch":209,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:42.483393+0000","last_change":"2023-06-27T12:41:42.492374+0000","last_active":"2023-06-27T12:41:42.483393+0000","last_peered":"2023-06-27T12:41:42.483393+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:42.483393+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":1,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":24673,"num_write":9,"num_write_kb":2549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"209'270","reported_seq":656,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-06-27T12:41:42.804237+0000","last_change":"2023-06-27T12:41:42.492884+0000","last_active":"2023-06-27T12:41:42.804237+0000","last_peered":"2023-06-27T12:41:42.804237+0000","last_clean":"2023-06-27T12:41:42.804237+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:42.804237+0000","last_undegraded":"2023-06-27T12:41:42.804237+0000","last_fullsized":"2023-06-27T12:41:42.804237+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12790424,"num_objects":31,"num_object_clones":25,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":208,"num_read_kb":134857,"num_write":153,"num_write_kb":43541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"205'143","reported_seq":533,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-06-27T12:41:42.479956+0000","last_change":"2023-06-27T12:41:42.506911+0000","last_active":"2023-06-27T12:41:42.479956+0000","last_peered":"2023-06-27T12:41:42.479956+0000","last_clean":"2023-06-27T12:41:42.479956+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:42.479956+0000","last_undegraded":"2023-06-27T12:41:42.479956+0000","last_fullsized":"2023-06-27T12:41:42.479956+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5748093,"num_objects":17,"num_object_clones":14,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":87,"num_read_kb":54982,"num_write":80,"num_write_kb":23093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"199'60","reported_seq":562,"reported_epoch":210,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:43.465733+0000","last_change":"2023-06-27T12:41:42.479522+0000","last_active":"2023-06-27T12:41:43.465733+0000","last_peered":"2023-06-27T12:41:43.465733+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:43.465733+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916064,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":340,"reported_epoch":209,"state":"active+undersized","last_fresh":"2023-06-27T12:41:42.457847+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:42.457847+0000","last_peered":"2023-06-27T12:41:42.457847+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:42.457847+0000","last_undegraded":"2023-06-27T12:41:42.457847+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":484,"reported_epoch":210,"state":"active+undersized","last_fresh":"2023-06-27T12:41:43.465199+0000","last_change":"2023-06-27T12:41:42.487871+0000","last_active":"2023-06-27T12:41:43.465199+0000","last_peered":"2023-06-27T12:41:43.465199+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:43.465199+0000","last_undegraded":"2023-06-27T12:41:43.465199+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":317,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-06-27T12:41:42.487881+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:42.487881+0000","last_peered":"2023-06-27T12:41:42.487881+0000","last_clean":"2023-06-27T12:41:42.487881+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:42.487881+0000","last_undegraded":"2023-06-27T12:41:42.487881+0000","last_fullsized":"2023-06-27T12:41:42.487881+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"205'109","reported_seq":613,"reported_epoch":209,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:42.463183+0000","last_change":"2023-06-27T12:41:42.463183+0000","last_active":"2023-06-27T12:41:42.463183+0000","last_peered":"2023-06-27T12:41:42.463183+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:42.463183+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2269011,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":56,"num_read_kb":56876,"num_write":68,"num_write_kb":18284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":13}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":326,"reported_epoch":210,"state":"active+undersized","last_fresh":"2023-06-27T12:41:43.464162+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:43.464162+0000","last_peered":"2023-06-27T12:41:43.464162+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:43.464162+0000","last_undegraded":"2023-06-27T12:41:43.464162+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"205'300","reported_seq":245,"reported_epoch":209,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:42.459168+0000","last_change":"2023-06-27T12:41:42.464539+0000","last_active":"2023-06-27T12:41:42.459168+0000","last_peered":"2023-06-27T12:41:42.459168+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:42.459168+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1210203,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":24,"num_read_kb":12413,"num_write":42,"num_write_kb":12514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":323,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-06-27T12:41:43.464420+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:43.464420+0000","last_peered":"2023-06-27T12:41:43.464420+0000","last_clean":"2023-06-27T12:41:43.464420+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:43.464420+0000","last_undegraded":"2023-06-27T12:41:43.464420+0000","last_fullsized":"2023-06-27T12:41:43.464420+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":69,"reported_epoch":209,"state":"active+undersized","last_fresh":"2023-06-27T12:41:42.481754+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:42.481754+0000","last_peered":"2023-06-27T12:41:42.481754+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:42.481754+0000","last_undegraded":"2023-06-27T12:41:42.481754+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"208'242","reported_seq":768,"reported_epoch":210,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:43.463506+0000","last_change":"2023-06-27T12:41:42.553501+0000","last_active":"2023-06-27T12:41:43.463506+0000","last_peered":"2023-06-27T12:41:43.463506+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:43.463506+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14487832,"num_objects":33,"num_object_clones":27,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":206,"num_read_kb":124638,"num_write":174,"num_write_kb":49962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":33}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":317,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-06-27T12:41:42.487430+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:42.487430+0000","last_peered":"2023-06-27T12:41:42.487430+0000","last_clean":"2023-06-27T12:41:42.487430+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:42.487430+0000","last_undegraded":"2023-06-27T12:41:42.487430+0000","last_fullsized":"2023-06-27T12:41:42.487430+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"209'132","reported_seq":215,"reported_epoch":210,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:43.466027+0000","last_change":"2023-06-27T12:41:42.487463+0000","last_active":"2023-06-27T12:41:43.466027+0000","last_peered":"2023-06-27T12:41:43.466027+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:43.466027+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3781740,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":56,"num_read_kb":35682,"num_write":25,"num_write_kb":7384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":209,"state":"active+undersized","last_fresh":"2023-06-27T12:41:42.457253+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:42.457253+0000","last_peered":"2023-06-27T12:41:42.457253+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:42.457253+0000","last_undegraded":"2023-06-27T12:41:42.457253+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"204'122","reported_seq":637,"reported_epoch":209,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:42.453082+0000","last_change":"2023-06-27T12:41:42.461150+0000","last_active":"2023-06-27T12:41:42.453082+0000","last_peered":"2023-06-27T12:41:42.453082+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:42.453082+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8353211,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":71,"num_read_kb":71931,"num_write":86,"num_write_kb":25844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"208'81","reported_seq":206,"reported_epoch":210,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:43.465441+0000","last_change":"2023-06-27T12:41:42.459817+0000","last_active":"2023-06-27T12:41:43.465441+0000","last_peered":"2023-06-27T12:41:43.465441+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:43.465441+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6386759,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":21761,"num_write":18,"num_write_kb":5277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":340,"reported_epoch":209,"state":"active+undersized","last_fresh":"2023-06-27T12:41:42.454780+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:42.454780+0000","last_peered":"2023-06-27T12:41:42.454780+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:42.454780+0000","last_undegraded":"2023-06-27T12:41:42.454780+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":323,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-06-27T12:41:43.470605+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:43.470605+0000","last_peered":"2023-06-27T12:41:43.470605+0000","last_clean":"2023-06-27T12:41:43.470605+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:43.470605+0000","last_undegraded":"2023-06-27T12:41:43.470605+0000","last_fullsized":"2023-06-27T12:41:43.470605+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"201'127","reported_seq":615,"reported_epoch":209,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:42.453724+0000","last_change":"2023-06-27T12:41:42.460552+0000","last_active":"2023-06-27T12:41:42.453724+0000","last_peered":"2023-06-27T12:41:42.453724+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:42.453724+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117715,"num_objects":13,"num_object_clones":10,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":62,"num_read_kb":60134,"num_write":57,"num_write_kb":16091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":13}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":79683505,"num_objects":199,"num_object_clones":157,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":1166,"num_read_kb":821588,"num_write":997,"num_write_kb":288767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1981,"ondisk_log_size":1981,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":169440,"kb_used_data":137568,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94202400,"statfs":{"total":96636764160,"available":96463257600,"internally_reserved":0,"allocated":140869632,"data_stored":160943915,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":186304,"kb_used_data":153460,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94185536,"statfs":{"total":96636764160,"available":96445988864,"internally_reserved":0,"allocated":157143040,"data_stored":177848834,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":200336,"kb_used_data":169416,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94171504,"statfs":{"total":96636764160,"available":96431620096,"internally_reserved":0,"allocated":173481984,"data_stored":197046376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:48.871 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.c 3.b 3.8 3.3 3.0 3.1 3.4 2023-06-27T12:41:49.047 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:49.031+0000 7f16ca925700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:49.049 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:49.045+0000 7f16ca925700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:49.244 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.c,3.f] on osd.0 to force-recovery; instructing pg(s) [3.1,3.8] on osd.1 to force-recovery; instructing pg(s) [3.0,3.3,3.4] on osd.2 to force-recovery; 2023-06-27T12:41:49.244 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:41:49.468 INFO:tasks.rados.rados.0.smithi138.stdout:827: done (2 left) 2023-06-27T12:41:49.468 INFO:tasks.rados.rados.0.smithi138.stdout:829: done (1 left) 2023-06-27T12:41:49.469 INFO:tasks.rados.rados.0.smithi138.stdout:830: done (0 left) 2023-06-27T12:41:49.469 INFO:tasks.rados.rados.0.smithi138.stdout:831: read oid 3 snap 96 2023-06-27T12:41:49.469 INFO:tasks.rados.rados.0.smithi138.stdout:831: expect deleted 2023-06-27T12:41:49.469 INFO:tasks.rados.rados.0.smithi138.stdout:832: write oid 49 current snap is 97 2023-06-27T12:41:49.469 INFO:tasks.rados.rados.0.smithi138.stdout:832: seq_num 400 ranges {722800=549956,2040562=682238} 2023-06-27T12:41:49.475 INFO:tasks.rados.rados.0.smithi138.stdout:832: writing smithi138138488-49 from 722800 to 1272756 tid 1 2023-06-27T12:41:49.485 INFO:tasks.rados.rados.0.smithi138.stdout:832: writing smithi138138488-49 from 2040562 to 2722800 tid 2 2023-06-27T12:41:49.485 INFO:tasks.rados.rados.0.smithi138.stdout:833: read oid 17 snap -1 2023-06-27T12:41:49.486 INFO:tasks.rados.rados.0.smithi138.stdout:833: expect (ObjNum 321 snap 81 seq_num 321) 2023-06-27T12:41:49.486 INFO:tasks.rados.rados.0.smithi138.stdout:834: snap_create 2023-06-27T12:41:49.516 INFO:tasks.rados.rados.0.smithi138.stdout:833: expect (ObjNum 321 snap 81 seq_num 321) 2023-06-27T12:41:49.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:49.762 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:49.757+0000 7faa563f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:49.765 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:49.761+0000 7faa563f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:49.783 INFO:tasks.rados.rados.0.smithi138.stdout:832: finishing write tid 1 to smithi138138488-49 2023-06-27T12:41:49.783 INFO:tasks.rados.rados.0.smithi138.stdout:832: finishing write tid 2 to smithi138138488-49 2023-06-27T12:41:49.783 INFO:tasks.rados.rados.0.smithi138.stdout:832: finishing write tid 3 to smithi138138488-49 2023-06-27T12:41:49.783 INFO:tasks.rados.rados.0.smithi138.stdout:832: finishing write tid 4 to smithi138138488-49 2023-06-27T12:41:49.783 INFO:tasks.rados.rados.0.smithi138.stdout:832: oid 49 updating version 0 to 136 2023-06-27T12:41:49.784 INFO:tasks.rados.rados.0.smithi138.stdout:832: oid 49 version 136 is already newer than 136 2023-06-27T12:41:49.784 INFO:tasks.rados.rados.0.smithi138.stdout:832: oid 49 version 136 is already newer than 136 2023-06-27T12:41:49.784 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 49 v 136 (ObjNum 400 snap 97 seq_num 400) dirty exists 2023-06-27T12:41:49.784 INFO:tasks.rados.rados.0.smithi138.stdout:832: left oid 49 (ObjNum 400 snap 97 seq_num 400) 2023-06-27T12:41:49.810 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:49.813 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:49.813 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:50.474 INFO:tasks.rados.rados.0.smithi138.stdout:831: done (3 left) 2023-06-27T12:41:50.475 INFO:tasks.rados.rados.0.smithi138.stdout:832: done (2 left) 2023-06-27T12:41:50.475 INFO:tasks.rados.rados.0.smithi138.stdout:833: done (1 left) 2023-06-27T12:41:50.475 INFO:tasks.rados.rados.0.smithi138.stdout:834: done (0 left) 2023-06-27T12:41:50.475 INFO:tasks.rados.rados.0.smithi138.stdout:835: read oid 1 snap -1 2023-06-27T12:41:50.475 INFO:tasks.rados.rados.0.smithi138.stdout:835: expect (ObjNum 341 snap 84 seq_num 341) 2023-06-27T12:41:50.475 INFO:tasks.rados.rados.0.smithi138.stdout:836: read oid 43 snap -1 2023-06-27T12:41:50.476 INFO:tasks.rados.rados.0.smithi138.stdout:836: expect (ObjNum 345 snap 86 seq_num 345) 2023-06-27T12:41:50.476 INFO:tasks.rados.rados.0.smithi138.stdout:837: read oid 41 snap -1 2023-06-27T12:41:50.476 INFO:tasks.rados.rados.0.smithi138.stdout:837: expect (ObjNum 268 snap 70 seq_num 268) 2023-06-27T12:41:50.476 INFO:tasks.rados.rados.0.smithi138.stdout:838: write (excl) oid 25 current snap is 98 2023-06-27T12:41:50.476 INFO:tasks.rados.rados.0.smithi138.stdout:838: seq_num 401 ranges {739607=625205,1982691=499457} 2023-06-27T12:41:50.483 INFO:tasks.rados.rados.0.smithi138.stdout:838: writing smithi138138488-25 from 739607 to 1364812 tid 1 2023-06-27T12:41:50.491 INFO:tasks.rados.rados.0.smithi138.stdout:838: writing smithi138138488-25 from 1982691 to 2482148 tid 2 2023-06-27T12:41:50.492 INFO:tasks.rados.rados.0.smithi138.stdout:839: delete oid 13 current snap is 98 2023-06-27T12:41:50.506 INFO:tasks.rados.rados.0.smithi138.stdout:839: done (4 left) 2023-06-27T12:41:50.506 INFO:tasks.rados.rados.0.smithi138.stdout:840: read oid 8 snap 96 2023-06-27T12:41:50.506 INFO:tasks.rados.rados.0.smithi138.stdout:840: expect deleted 2023-06-27T12:41:50.507 INFO:tasks.rados.rados.0.smithi138.stdout:841: read oid 21 snap -1 2023-06-27T12:41:50.507 INFO:tasks.rados.rados.0.smithi138.stdout:841: expect (ObjNum 265 snap 70 seq_num 265) 2023-06-27T12:41:50.507 INFO:tasks.rados.rados.0.smithi138.stdout:842: read oid 13 snap 97 2023-06-27T12:41:50.507 INFO:tasks.rados.rados.0.smithi138.stdout:842: expect deleted 2023-06-27T12:41:50.507 INFO:tasks.rados.rados.0.smithi138.stdout:843: delete oid 45 current snap is 98 2023-06-27T12:41:50.527 INFO:tasks.rados.rados.0.smithi138.stdout:836: expect (ObjNum 345 snap 86 seq_num 345) 2023-06-27T12:41:50.722 INFO:tasks.rados.rados.0.smithi138.stdout:837: expect (ObjNum 268 snap 70 seq_num 268) 2023-06-27T12:41:50.889 INFO:tasks.rados.rados.0.smithi138.stdout:835: expect (ObjNum 341 snap 84 seq_num 341) 2023-06-27T12:41:51.027 INFO:tasks.rados.rados.0.smithi138.stdout:838: finishing write tid 1 to smithi138138488-25 2023-06-27T12:41:51.028 INFO:tasks.rados.rados.0.smithi138.stdout:841: expect (ObjNum 265 snap 70 seq_num 265) 2023-06-27T12:41:51.155 INFO:tasks.rados.rados.0.smithi138.stdout:838: finishing write tid 2 to smithi138138488-25 2023-06-27T12:41:51.155 INFO:tasks.rados.rados.0.smithi138.stdout:838: finishing write tid 3 to smithi138138488-25 2023-06-27T12:41:51.156 INFO:tasks.rados.rados.0.smithi138.stdout:838: finishing write tid 4 to smithi138138488-25 2023-06-27T12:41:51.156 INFO:tasks.rados.rados.0.smithi138.stdout:838: oid 25 updating version 0 to 89 2023-06-27T12:41:51.156 INFO:tasks.rados.rados.0.smithi138.stdout:838: oid 25 version 89 is already newer than 89 2023-06-27T12:41:51.156 INFO:tasks.rados.rados.0.smithi138.stdout:838: oid 25 version 89 is already newer than 86 2023-06-27T12:41:51.156 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 89 (ObjNum 401 snap 98 seq_num 401) dirty exists 2023-06-27T12:41:51.156 INFO:tasks.rados.rados.0.smithi138.stdout:838: left oid 25 (ObjNum 401 snap 98 seq_num 401) 2023-06-27T12:41:51.157 INFO:tasks.rados.rados.0.smithi138.stdout:835: done (7 left) 2023-06-27T12:41:51.157 INFO:tasks.rados.rados.0.smithi138.stdout:836: done (6 left) 2023-06-27T12:41:51.157 INFO:tasks.rados.rados.0.smithi138.stdout:837: done (5 left) 2023-06-27T12:41:51.157 INFO:tasks.rados.rados.0.smithi138.stdout:838: done (4 left) 2023-06-27T12:41:51.157 INFO:tasks.rados.rados.0.smithi138.stdout:840: done (3 left) 2023-06-27T12:41:51.157 INFO:tasks.rados.rados.0.smithi138.stdout:841: done (2 left) 2023-06-27T12:41:51.157 INFO:tasks.rados.rados.0.smithi138.stdout:842: done (1 left) 2023-06-27T12:41:51.158 INFO:tasks.rados.rados.0.smithi138.stdout:843: done (0 left) 2023-06-27T12:41:51.158 INFO:tasks.rados.rados.0.smithi138.stdout:844: snap_remove snap 96 2023-06-27T12:41:51.483 INFO:tasks.rados.rados.0.smithi138.stdout:844: done (0 left) 2023-06-27T12:41:51.483 INFO:tasks.rados.rados.0.smithi138.stdout:845: write oid 45 current snap is 98 2023-06-27T12:41:51.484 INFO:tasks.rados.rados.0.smithi138.stdout:845: seq_num 404 ranges {790028=567305,1874490=763137} 2023-06-27T12:41:51.493 INFO:tasks.rados.rados.0.smithi138.stdout:845: writing smithi138138488-45 from 790028 to 1357333 tid 1 2023-06-27T12:41:51.503 INFO:tasks.rados.rados.0.smithi138.stdout:845: writing smithi138138488-45 from 1874490 to 2637627 tid 2 2023-06-27T12:41:51.503 INFO:tasks.rados.rados.0.smithi138.stdout:846: write oid 46 current snap is 98 2023-06-27T12:41:51.503 INFO:tasks.rados.rados.0.smithi138.stdout:846: seq_num 405 ranges {406835=642554,1532972=506795,2772312=34523} 2023-06-27T12:41:51.511 INFO:tasks.rados.rados.0.smithi138.stdout:846: writing smithi138138488-46 from 406835 to 1049389 tid 1 2023-06-27T12:41:51.518 INFO:tasks.rados.rados.0.smithi138.stdout:846: writing smithi138138488-46 from 1532972 to 2039767 tid 2 2023-06-27T12:41:51.522 INFO:tasks.rados.rados.0.smithi138.stdout:846: writing smithi138138488-46 from 2772312 to 2806835 tid 3 2023-06-27T12:41:51.523 INFO:tasks.rados.rados.0.smithi138.stdout:847: read oid 13 snap 97 2023-06-27T12:41:51.523 INFO:tasks.rados.rados.0.smithi138.stdout:847: expect deleted 2023-06-27T12:41:51.523 INFO:tasks.rados.rados.0.smithi138.stdout:848: write (excl) oid 1 current snap is 98 2023-06-27T12:41:51.523 INFO:tasks.rados.rados.0.smithi138.stdout:848: seq_num 406 ranges {423642=717803,1875101=766806} 2023-06-27T12:41:51.529 INFO:tasks.rados.rados.0.smithi138.stdout:848: writing smithi138138488-1 from 423642 to 1141445 tid 1 2023-06-27T12:41:51.541 INFO:tasks.rados.rados.0.smithi138.stdout:848: writing smithi138138488-1 from 1875101 to 2641907 tid 2 2023-06-27T12:41:51.541 INFO:tasks.rados.rados.0.smithi138.stdout:849: read oid 19 snap 97 2023-06-27T12:41:51.541 INFO:tasks.rados.rados.0.smithi138.stdout:849: expect (ObjNum 388 snap 95 seq_num 388) 2023-06-27T12:41:51.541 INFO:tasks.rados.rados.0.smithi138.stdout:850: write (excl) oid 42 current snap is 98 2023-06-27T12:41:51.541 INFO:tasks.rados.rados.0.smithi138.stdout:850: seq_num 407 ranges {440449=793052,1933583=510464} 2023-06-27T12:41:51.548 INFO:tasks.rados.rados.0.smithi138.stdout:850: writing smithi138138488-42 from 440449 to 1233501 tid 1 2023-06-27T12:41:51.555 INFO:tasks.rados.rados.0.smithi138.stdout:850: writing smithi138138488-42 from 1933583 to 2444047 tid 2 2023-06-27T12:41:51.555 INFO:tasks.rados.rados.0.smithi138.stdout:851: delete oid 29 current snap is 98 2023-06-27T12:41:51.555 INFO:tasks.rados.rados.0.smithi138.stdout:845: finishing write tid 1 to smithi138138488-45 2023-06-27T12:41:51.566 INFO:tasks.rados.rados.0.smithi138.stdout:845: finishing write tid 2 to smithi138138488-45 2023-06-27T12:41:51.579 INFO:tasks.rados.rados.0.smithi138.stdout:846: finishing write tid 1 to smithi138138488-46 2023-06-27T12:41:51.581 INFO:tasks.rados.rados.0.smithi138.stdout:848: finishing write tid 1 to smithi138138488-1 2023-06-27T12:41:51.620 INFO:tasks.rados.rados.0.smithi138.stdout:846: finishing write tid 2 to smithi138138488-46 2023-06-27T12:41:51.631 INFO:tasks.rados.rados.0.smithi138.stdout:846: finishing write tid 3 to smithi138138488-46 2023-06-27T12:41:51.632 INFO:tasks.rados.rados.0.smithi138.stdout:846: finishing write tid 4 to smithi138138488-46 2023-06-27T12:41:51.633 INFO:tasks.rados.rados.0.smithi138.stdout:846: finishing write tid 5 to smithi138138488-46 2023-06-27T12:41:51.633 INFO:tasks.rados.rados.0.smithi138.stdout:846: oid 46 updating version 0 to 118 2023-06-27T12:41:51.634 INFO:tasks.rados.rados.0.smithi138.stdout:846: oid 46 version 118 is already newer than 118 2023-06-27T12:41:51.634 INFO:tasks.rados.rados.0.smithi138.stdout:846: oid 46 version 118 is already newer than 118 2023-06-27T12:41:51.634 INFO:tasks.rados.rados.0.smithi138.stdout:846: oid 46 version 118 is already newer than 118 2023-06-27T12:41:51.634 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 118 (ObjNum 405 snap 98 seq_num 405) dirty exists 2023-06-27T12:41:51.634 INFO:tasks.rados.rados.0.smithi138.stdout:846: left oid 46 (ObjNum 405 snap 98 seq_num 405) 2023-06-27T12:41:51.639 INFO:tasks.rados.rados.0.smithi138.stdout:850: finishing write tid 1 to smithi138138488-42 2023-06-27T12:41:51.665 INFO:tasks.rados.rados.0.smithi138.stdout:848: finishing write tid 2 to smithi138138488-1 2023-06-27T12:41:51.665 INFO:tasks.rados.rados.0.smithi138.stdout:846: done (6 left) 2023-06-27T12:41:51.665 INFO:tasks.rados.rados.0.smithi138.stdout:851: done (5 left) 2023-06-27T12:41:51.665 INFO:tasks.rados.rados.0.smithi138.stdout:852: delete oid 17 current snap is 98 2023-06-27T12:41:51.666 INFO:tasks.rados.rados.0.smithi138.stdout:848: finishing write tid 3 to smithi138138488-1 2023-06-27T12:41:51.667 INFO:tasks.rados.rados.0.smithi138.stdout:848: finishing write tid 4 to smithi138138488-1 2023-06-27T12:41:51.667 INFO:tasks.rados.rados.0.smithi138.stdout:848: oid 1 updating version 0 to 251 2023-06-27T12:41:51.667 INFO:tasks.rados.rados.0.smithi138.stdout:848: oid 1 version 251 is already newer than 248 2023-06-27T12:41:51.667 INFO:tasks.rados.rados.0.smithi138.stdout:848: oid 1 version 251 is already newer than 251 2023-06-27T12:41:51.668 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 251 (ObjNum 406 snap 98 seq_num 406) dirty exists 2023-06-27T12:41:51.668 INFO:tasks.rados.rados.0.smithi138.stdout:848: left oid 1 (ObjNum 406 snap 98 seq_num 406) 2023-06-27T12:41:51.671 INFO:tasks.rados.rados.0.smithi138.stdout:845: finishing write tid 3 to smithi138138488-45 2023-06-27T12:41:51.685 INFO:tasks.rados.rados.0.smithi138.stdout:849: expect (ObjNum 388 snap 95 seq_num 388) 2023-06-27T12:41:51.814 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:51.848 INFO:tasks.rados.rados.0.smithi138.stdout:850: finishing write tid 2 to smithi138138488-42 2023-06-27T12:41:51.848 INFO:tasks.rados.rados.0.smithi138.stdout:848: done (5 left) 2023-06-27T12:41:51.848 INFO:tasks.rados.rados.0.smithi138.stdout:849: done (4 left) 2023-06-27T12:41:51.848 INFO:tasks.rados.rados.0.smithi138.stdout:852: done (3 left) 2023-06-27T12:41:51.849 INFO:tasks.rados.rados.0.smithi138.stdout:853: delete oid 3 current snap is 98 2023-06-27T12:41:51.849 INFO:tasks.rados.rados.0.smithi138.stdout:850: finishing write tid 3 to smithi138138488-42 2023-06-27T12:41:51.849 INFO:tasks.rados.rados.0.smithi138.stdout:850: finishing write tid 4 to smithi138138488-42 2023-06-27T12:41:51.849 INFO:tasks.rados.rados.0.smithi138.stdout:850: oid 42 updating version 0 to 80 2023-06-27T12:41:51.849 INFO:tasks.rados.rados.0.smithi138.stdout:850: oid 42 updating version 80 to 82 2023-06-27T12:41:51.850 INFO:tasks.rados.rados.0.smithi138.stdout:850: oid 42 version 82 is already newer than 82 2023-06-27T12:41:51.850 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 82 (ObjNum 407 snap 98 seq_num 407) dirty exists 2023-06-27T12:41:51.850 INFO:tasks.rados.rados.0.smithi138.stdout:850: left oid 42 (ObjNum 407 snap 98 seq_num 407) 2023-06-27T12:41:51.850 INFO:tasks.rados.rados.0.smithi138.stdout:845: finishing write tid 4 to smithi138138488-45 2023-06-27T12:41:51.850 INFO:tasks.rados.rados.0.smithi138.stdout:845: oid 45 updating version 0 to 308 2023-06-27T12:41:51.850 INFO:tasks.rados.rados.0.smithi138.stdout:845: oid 45 updating version 308 to 311 2023-06-27T12:41:51.850 INFO:tasks.rados.rados.0.smithi138.stdout:845: oid 45 version 311 is already newer than 308 2023-06-27T12:41:51.851 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 311 (ObjNum 404 snap 98 seq_num 404) dirty exists 2023-06-27T12:41:51.851 INFO:tasks.rados.rados.0.smithi138.stdout:845: left oid 45 (ObjNum 404 snap 98 seq_num 404) 2023-06-27T12:41:51.851 INFO:tasks.rados.rados.0.smithi138.stdout:845: done (3 left) 2023-06-27T12:41:51.852 INFO:tasks.rados.rados.0.smithi138.stdout:847: done (2 left) 2023-06-27T12:41:51.852 INFO:tasks.rados.rados.0.smithi138.stdout:850: done (1 left) 2023-06-27T12:41:51.852 INFO:tasks.rados.rados.0.smithi138.stdout:853: done (0 left) 2023-06-27T12:41:51.852 INFO:tasks.rados.rados.0.smithi138.stdout:854: read oid 22 snap -1 2023-06-27T12:41:51.853 INFO:tasks.rados.rados.0.smithi138.stdout:854: expect deleted 2023-06-27T12:41:51.853 INFO:tasks.rados.rados.0.smithi138.stdout:855: snap_remove snap 97 2023-06-27T12:41:51.944 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:51.939+0000 7f932be90700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:51.947 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:51.943+0000 7f932be90700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:51.991 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:51.994 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:51.995 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:52.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~77s 2023-06-27T12:41:52.490 INFO:tasks.rados.rados.0.smithi138.stdout:855: done (1 left) 2023-06-27T12:41:52.490 INFO:tasks.rados.rados.0.smithi138.stdout:856: delete oid 3 current snap is 98 2023-06-27T12:41:52.524 INFO:tasks.rados.rados.0.smithi138.stdout:854: done (1 left) 2023-06-27T12:41:52.524 INFO:tasks.rados.rados.0.smithi138.stdout:856: done (0 left) 2023-06-27T12:41:52.524 INFO:tasks.rados.rados.0.smithi138.stdout:857: delete oid 16 current snap is 98 2023-06-27T12:41:52.580 INFO:tasks.rados.rados.0.smithi138.stdout:857: done (0 left) 2023-06-27T12:41:52.581 INFO:tasks.rados.rados.0.smithi138.stdout:858: write oid 4 current snap is 98 2023-06-27T12:41:52.581 INFO:tasks.rados.rados.0.smithi138.stdout:858: seq_num 413 ranges {541291=560899,1884475=521471} 2023-06-27T12:41:52.586 INFO:tasks.rados.rados.0.smithi138.stdout:858: writing smithi138138488-4 from 541291 to 1102190 tid 1 2023-06-27T12:41:52.596 INFO:tasks.rados.rados.0.smithi138.stdout:858: writing smithi138138488-4 from 1884475 to 2405946 tid 2 2023-06-27T12:41:52.596 INFO:tasks.rados.rados.0.smithi138.stdout:859: read oid 11 snap -1 2023-06-27T12:41:52.596 INFO:tasks.rados.rados.0.smithi138.stdout:859: expect (ObjNum 389 snap 95 seq_num 389) 2023-06-27T12:41:52.596 INFO:tasks.rados.rados.0.smithi138.stdout:860: read oid 44 snap -1 2023-06-27T12:41:52.596 INFO:tasks.rados.rados.0.smithi138.stdout:860: expect (ObjNum 384 snap 93 seq_num 384) 2023-06-27T12:41:52.596 INFO:tasks.rados.rados.0.smithi138.stdout:861: delete oid 40 current snap is 98 2023-06-27T12:41:52.602 INFO:tasks.rados.rados.0.smithi138.stdout:861: done (3 left) 2023-06-27T12:41:52.603 INFO:tasks.rados.rados.0.smithi138.stdout:862: read oid 13 snap -1 2023-06-27T12:41:52.603 INFO:tasks.rados.rados.0.smithi138.stdout:862: expect deleted 2023-06-27T12:41:52.603 INFO:tasks.rados.rados.0.smithi138.stdout:863: write (excl) oid 12 current snap is 98 2023-06-27T12:41:52.603 INFO:tasks.rados.rados.0.smithi138.stdout:863: seq_num 415 ranges {574905=711397,1885086=525140,2930189=44716} 2023-06-27T12:41:52.612 INFO:tasks.rados.rados.0.smithi138.stdout:863: writing smithi138138488-12 from 574905 to 1286302 tid 1 2023-06-27T12:41:52.619 INFO:tasks.rados.rados.0.smithi138.stdout:863: writing smithi138138488-12 from 1885086 to 2410226 tid 2 2023-06-27T12:41:52.623 INFO:tasks.rados.rados.0.smithi138.stdout:863: writing smithi138138488-12 from 2930189 to 2974905 tid 3 2023-06-27T12:41:52.623 INFO:tasks.rados.rados.0.smithi138.stdout:864: read oid 28 snap -1 2023-06-27T12:41:52.623 INFO:tasks.rados.rados.0.smithi138.stdout:864: expect (ObjNum 330 snap 83 seq_num 330) 2023-06-27T12:41:52.623 INFO:tasks.rados.rados.0.smithi138.stdout:865: snap_create 2023-06-27T12:41:52.623 INFO:tasks.rados.rados.0.smithi138.stdout:860: expect (ObjNum 384 snap 93 seq_num 384) 2023-06-27T12:41:52.804 INFO:tasks.rados.rados.0.smithi138.stdout:864: expect (ObjNum 330 snap 83 seq_num 330) 2023-06-27T12:41:53.063 INFO:tasks.rados.rados.0.smithi138.stdout:858: finishing write tid 1 to smithi138138488-4 2023-06-27T12:41:53.063 INFO:tasks.rados.rados.0.smithi138.stdout:858: finishing write tid 2 to smithi138138488-4 2023-06-27T12:41:53.063 INFO:tasks.rados.rados.0.smithi138.stdout:858: finishing write tid 3 to smithi138138488-4 2023-06-27T12:41:53.063 INFO:tasks.rados.rados.0.smithi138.stdout:858: finishing write tid 4 to smithi138138488-4 2023-06-27T12:41:53.064 INFO:tasks.rados.rados.0.smithi138.stdout:858: oid 4 updating version 0 to 139 2023-06-27T12:41:53.064 INFO:tasks.rados.rados.0.smithi138.stdout:858: oid 4 version 139 is already newer than 139 2023-06-27T12:41:53.064 INFO:tasks.rados.rados.0.smithi138.stdout:858: oid 4 version 139 is already newer than 139 2023-06-27T12:41:53.064 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 139 (ObjNum 413 snap 98 seq_num 413) dirty exists 2023-06-27T12:41:53.064 INFO:tasks.rados.rados.0.smithi138.stdout:858: left oid 4 (ObjNum 413 snap 98 seq_num 413) 2023-06-27T12:41:53.064 INFO:tasks.rados.rados.0.smithi138.stdout:859: expect (ObjNum 389 snap 95 seq_num 389) 2023-06-27T12:41:53.158 INFO:tasks.rados.rados.0.smithi138.stdout:863: finishing write tid 1 to smithi138138488-12 2023-06-27T12:41:53.159 INFO:tasks.rados.rados.0.smithi138.stdout:863: finishing write tid 2 to smithi138138488-12 2023-06-27T12:41:53.159 INFO:tasks.rados.rados.0.smithi138.stdout:863: finishing write tid 3 to smithi138138488-12 2023-06-27T12:41:53.159 INFO:tasks.rados.rados.0.smithi138.stdout:863: finishing write tid 4 to smithi138138488-12 2023-06-27T12:41:53.159 INFO:tasks.rados.rados.0.smithi138.stdout:863: finishing write tid 5 to smithi138138488-12 2023-06-27T12:41:53.159 INFO:tasks.rados.rados.0.smithi138.stdout:863: oid 12 updating version 0 to 169 2023-06-27T12:41:53.159 INFO:tasks.rados.rados.0.smithi138.stdout:863: oid 12 version 169 is already newer than 169 2023-06-27T12:41:53.160 INFO:tasks.rados.rados.0.smithi138.stdout:863: oid 12 version 169 is already newer than 169 2023-06-27T12:41:53.160 INFO:tasks.rados.rados.0.smithi138.stdout:863: oid 12 version 169 is already newer than 166 2023-06-27T12:41:53.160 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 169 (ObjNum 415 snap 98 seq_num 415) dirty exists 2023-06-27T12:41:53.160 INFO:tasks.rados.rados.0.smithi138.stdout:863: left oid 12 (ObjNum 415 snap 98 seq_num 415) 2023-06-27T12:41:53.491 INFO:tasks.rados.rados.0.smithi138.stdout:858: done (6 left) 2023-06-27T12:41:53.491 INFO:tasks.rados.rados.0.smithi138.stdout:859: done (5 left) 2023-06-27T12:41:53.492 INFO:tasks.rados.rados.0.smithi138.stdout:860: done (4 left) 2023-06-27T12:41:53.492 INFO:tasks.rados.rados.0.smithi138.stdout:862: done (3 left) 2023-06-27T12:41:53.492 INFO:tasks.rados.rados.0.smithi138.stdout:863: done (2 left) 2023-06-27T12:41:53.492 INFO:tasks.rados.rados.0.smithi138.stdout:864: done (1 left) 2023-06-27T12:41:53.492 INFO:tasks.rados.rados.0.smithi138.stdout:865: done (0 left) 2023-06-27T12:41:53.493 INFO:tasks.rados.rados.0.smithi138.stdout:866: delete oid 17 current snap is 99 2023-06-27T12:41:53.516 INFO:tasks.rados.rados.0.smithi138.stdout:866: done (0 left) 2023-06-27T12:41:53.516 INFO:tasks.rados.rados.0.smithi138.stdout:867: read oid 49 snap -1 2023-06-27T12:41:53.517 INFO:tasks.rados.rados.0.smithi138.stdout:867: expect (ObjNum 400 snap 97 seq_num 400) 2023-06-27T12:41:53.517 INFO:tasks.rados.rados.0.smithi138.stdout:868: delete oid 28 current snap is 99 2023-06-27T12:41:53.552 INFO:tasks.rados.rados.0.smithi138.stdout:867: expect (ObjNum 400 snap 97 seq_num 400) 2023-06-27T12:41:53.708 INFO:tasks.rados.rados.0.smithi138.stdout:867: done (1 left) 2023-06-27T12:41:53.708 INFO:tasks.rados.rados.0.smithi138.stdout:868: done (0 left) 2023-06-27T12:41:53.708 INFO:tasks.rados.rados.0.smithi138.stdout:869: read oid 11 snap -1 2023-06-27T12:41:53.708 INFO:tasks.rados.rados.0.smithi138.stdout:869: expect (ObjNum 389 snap 95 seq_num 389) 2023-06-27T12:41:53.708 INFO:tasks.rados.rados.0.smithi138.stdout:870: write oid 48 current snap is 99 2023-06-27T12:41:53.708 INFO:tasks.rados.rados.0.smithi138.stdout:870: seq_num 418 ranges {625326=537144,1944179=672467} 2023-06-27T12:41:53.714 INFO:tasks.rados.rados.0.smithi138.stdout:870: writing smithi138138488-48 from 625326 to 1162470 tid 1 2023-06-27T12:41:53.722 INFO:tasks.rados.rados.0.smithi138.stdout:870: writing smithi138138488-48 from 1944179 to 2616646 tid 2 2023-06-27T12:41:53.723 INFO:tasks.rados.rados.0.smithi138.stdout:871: write (excl) oid 27 current snap is 99 2023-06-27T12:41:53.723 INFO:tasks.rados.rados.0.smithi138.stdout:871: seq_num 419 ranges {642133=728746,2119014=532478} 2023-06-27T12:41:53.730 INFO:tasks.rados.rados.0.smithi138.stdout:871: writing smithi138138488-27 from 642133 to 1370879 tid 1 2023-06-27T12:41:53.737 INFO:tasks.rados.rados.0.smithi138.stdout:871: writing smithi138138488-27 from 2119014 to 2651492 tid 2 2023-06-27T12:41:53.737 INFO:tasks.rados.rados.0.smithi138.stdout:872: write oid 17 current snap is 99 2023-06-27T12:41:53.738 INFO:tasks.rados.rados.0.smithi138.stdout:872: seq_num 420 ranges {658940=403995,1777496=676136,2950951=107989} 2023-06-27T12:41:53.743 INFO:tasks.rados.rados.0.smithi138.stdout:872: writing smithi138138488-17 from 658940 to 1062935 tid 1 2023-06-27T12:41:53.751 INFO:tasks.rados.rados.0.smithi138.stdout:872: writing smithi138138488-17 from 1777496 to 2453632 tid 2 2023-06-27T12:41:53.755 INFO:tasks.rados.rados.0.smithi138.stdout:872: writing smithi138138488-17 from 2950951 to 3058940 tid 3 2023-06-27T12:41:53.755 INFO:tasks.rados.rados.0.smithi138.stdout:873: write (excl) oid 4 current snap is 99 2023-06-27T12:41:53.755 INFO:tasks.rados.rados.0.smithi138.stdout:873: seq_num 421 ranges {675747=479244,1835978=536147} 2023-06-27T12:41:53.761 INFO:tasks.rados.rados.0.smithi138.stdout:873: writing smithi138138488-4 from 675747 to 1154991 tid 1 2023-06-27T12:41:53.768 INFO:tasks.rados.rados.0.smithi138.stdout:873: writing smithi138138488-4 from 1835978 to 2372125 tid 2 2023-06-27T12:41:53.768 INFO:tasks.rados.rados.0.smithi138.stdout:874: write (excl) oid 47 current snap is 99 2023-06-27T12:41:53.768 INFO:tasks.rados.rados.0.smithi138.stdout:874: seq_num 422 ranges {692554=554493,1778107=679805,2889444=203110} 2023-06-27T12:41:53.775 INFO:tasks.rados.rados.0.smithi138.stdout:874: writing smithi138138488-47 from 692554 to 1247047 tid 1 2023-06-27T12:41:53.782 INFO:tasks.rados.rados.0.smithi138.stdout:874: writing smithi138138488-47 from 1778107 to 2457912 tid 2 2023-06-27T12:41:53.786 INFO:tasks.rados.rados.0.smithi138.stdout:874: writing smithi138138488-47 from 2889444 to 3092554 tid 3 2023-06-27T12:41:53.786 INFO:tasks.rados.rados.0.smithi138.stdout:875: write oid 3 current snap is 99 2023-06-27T12:41:53.787 INFO:tasks.rados.rados.0.smithi138.stdout:869: expect (ObjNum 389 snap 95 seq_num 389) 2023-06-27T12:41:53.892 INFO:tasks.rados.rados.0.smithi138.stdout:870: finishing write tid 1 to smithi138138488-48 2023-06-27T12:41:53.893 INFO:tasks.rados.rados.0.smithi138.stdout:870: finishing write tid 2 to smithi138138488-48 2023-06-27T12:41:53.893 INFO:tasks.rados.rados.0.smithi138.stdout:870: finishing write tid 3 to smithi138138488-48 2023-06-27T12:41:53.893 INFO:tasks.rados.rados.0.smithi138.stdout:870: finishing write tid 4 to smithi138138488-48 2023-06-27T12:41:53.893 INFO:tasks.rados.rados.0.smithi138.stdout:870: oid 48 updating version 0 to 141 2023-06-27T12:41:53.893 INFO:tasks.rados.rados.0.smithi138.stdout:870: oid 48 version 141 is already newer than 141 2023-06-27T12:41:53.893 INFO:tasks.rados.rados.0.smithi138.stdout:870: oid 48 version 141 is already newer than 141 2023-06-27T12:41:53.894 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 141 (ObjNum 418 snap 99 seq_num 418) dirty exists 2023-06-27T12:41:53.894 INFO:tasks.rados.rados.0.smithi138.stdout:870: left oid 48 (ObjNum 418 snap 99 seq_num 418) 2023-06-27T12:41:53.894 INFO:tasks.rados.rados.0.smithi138.stdout:871: finishing write tid 1 to smithi138138488-27 2023-06-27T12:41:53.894 INFO:tasks.rados.rados.0.smithi138.stdout:875: seq_num 423 ranges {709361=629742,1836589=539816,3033220=76141} 2023-06-27T12:41:53.900 INFO:tasks.rados.rados.0.smithi138.stdout:875: writing smithi138138488-3 from 709361 to 1339103 tid 1 2023-06-27T12:41:53.905 INFO:tasks.rados.rados.0.smithi138.stdout:875: writing smithi138138488-3 from 1836589 to 2376405 tid 2 2023-06-27T12:41:53.910 INFO:tasks.rados.rados.0.smithi138.stdout:875: writing smithi138138488-3 from 3033220 to 3109361 tid 3 2023-06-27T12:41:53.911 INFO:tasks.rados.rados.0.smithi138.stdout:869: done (6 left) 2023-06-27T12:41:53.911 INFO:tasks.rados.rados.0.smithi138.stdout:870: done (5 left) 2023-06-27T12:41:53.911 INFO:tasks.rados.rados.0.smithi138.stdout:876: read oid 16 snap -1 2023-06-27T12:41:53.911 INFO:tasks.rados.rados.0.smithi138.stdout:876: expect deleted 2023-06-27T12:41:53.911 INFO:tasks.rados.rados.0.smithi138.stdout:877: write (excl) oid 40 current snap is 99 2023-06-27T12:41:53.911 INFO:tasks.rados.rados.0.smithi138.stdout:873: finishing write tid 1 to smithi138138488-4 2023-06-27T12:41:53.911 INFO:tasks.rados.rados.0.smithi138.stdout:871: finishing write tid 2 to smithi138138488-27 2023-06-27T12:41:53.912 INFO:tasks.rados.rados.0.smithi138.stdout:873: finishing write tid 2 to smithi138138488-4 2023-06-27T12:41:53.912 INFO:tasks.rados.rados.0.smithi138.stdout:873: finishing write tid 3 to smithi138138488-4 2023-06-27T12:41:53.912 INFO:tasks.rados.rados.0.smithi138.stdout:873: finishing write tid 4 to smithi138138488-4 2023-06-27T12:41:53.912 INFO:tasks.rados.rados.0.smithi138.stdout:873: oid 4 updating version 0 to 143 2023-06-27T12:41:53.912 INFO:tasks.rados.rados.0.smithi138.stdout:873: oid 4 version 143 is already newer than 143 2023-06-27T12:41:53.912 INFO:tasks.rados.rados.0.smithi138.stdout:873: oid 4 version 143 is already newer than 140 2023-06-27T12:41:53.912 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 143 (ObjNum 421 snap 99 seq_num 421) dirty exists 2023-06-27T12:41:53.912 INFO:tasks.rados.rados.0.smithi138.stdout:873: left oid 4 (ObjNum 421 snap 99 seq_num 421) 2023-06-27T12:41:53.913 INFO:tasks.rados.rados.0.smithi138.stdout:871: finishing write tid 3 to smithi138138488-27 2023-06-27T12:41:53.913 INFO:tasks.rados.rados.0.smithi138.stdout:871: finishing write tid 4 to smithi138138488-27 2023-06-27T12:41:53.913 INFO:tasks.rados.rados.0.smithi138.stdout:871: oid 27 updating version 0 to 128 2023-06-27T12:41:53.913 INFO:tasks.rados.rados.0.smithi138.stdout:871: oid 27 version 128 is already newer than 126 2023-06-27T12:41:53.913 INFO:tasks.rados.rados.0.smithi138.stdout:871: oid 27 version 128 is already newer than 128 2023-06-27T12:41:53.913 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 128 (ObjNum 419 snap 99 seq_num 419) dirty exists 2023-06-27T12:41:53.913 INFO:tasks.rados.rados.0.smithi138.stdout:871: left oid 27 (ObjNum 419 snap 99 seq_num 419) 2023-06-27T12:41:53.913 INFO:tasks.rados.rados.0.smithi138.stdout:872: finishing write tid 1 to smithi138138488-17 2023-06-27T12:41:53.914 INFO:tasks.rados.rados.0.smithi138.stdout:872: finishing write tid 2 to smithi138138488-17 2023-06-27T12:41:53.914 INFO:tasks.rados.rados.0.smithi138.stdout:872: finishing write tid 3 to smithi138138488-17 2023-06-27T12:41:53.914 INFO:tasks.rados.rados.0.smithi138.stdout:872: finishing write tid 4 to smithi138138488-17 2023-06-27T12:41:53.914 INFO:tasks.rados.rados.0.smithi138.stdout:872: finishing write tid 5 to smithi138138488-17 2023-06-27T12:41:53.914 INFO:tasks.rados.rados.0.smithi138.stdout:872: oid 17 updating version 0 to 132 2023-06-27T12:41:53.914 INFO:tasks.rados.rados.0.smithi138.stdout:872: oid 17 version 132 is already newer than 132 2023-06-27T12:41:53.914 INFO:tasks.rados.rados.0.smithi138.stdout:872: oid 17 version 132 is already newer than 132 2023-06-27T12:41:53.914 INFO:tasks.rados.rados.0.smithi138.stdout:872: oid 17 version 132 is already newer than 132 2023-06-27T12:41:53.915 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 132 (ObjNum 420 snap 99 seq_num 420) dirty exists 2023-06-27T12:41:53.915 INFO:tasks.rados.rados.0.smithi138.stdout:872: left oid 17 (ObjNum 420 snap 99 seq_num 420) 2023-06-27T12:41:53.915 INFO:tasks.rados.rados.0.smithi138.stdout:874: finishing write tid 1 to smithi138138488-47 2023-06-27T12:41:53.915 INFO:tasks.rados.rados.0.smithi138.stdout:874: finishing write tid 2 to smithi138138488-47 2023-06-27T12:41:53.915 INFO:tasks.rados.rados.0.smithi138.stdout:877: seq_num 424 ranges {726168=704991,1895071=683474} 2023-06-27T12:41:53.918 INFO:tasks.rados.rados.0.smithi138.stdout:877: writing smithi138138488-40 from 726168 to 1431159 tid 1 2023-06-27T12:41:53.925 INFO:tasks.rados.rados.0.smithi138.stdout:877: writing smithi138138488-40 from 1895071 to 2578545 tid 2 2023-06-27T12:41:53.925 INFO:tasks.rados.rados.0.smithi138.stdout:871: done (6 left) 2023-06-27T12:41:53.925 INFO:tasks.rados.rados.0.smithi138.stdout:872: done (5 left) 2023-06-27T12:41:53.925 INFO:tasks.rados.rados.0.smithi138.stdout:873: done (4 left) 2023-06-27T12:41:53.926 INFO:tasks.rados.rados.0.smithi138.stdout:878: snap_create 2023-06-27T12:41:53.926 INFO:tasks.rados.rados.0.smithi138.stdout:875: finishing write tid 1 to smithi138138488-3 2023-06-27T12:41:53.936 INFO:tasks.rados.rados.0.smithi138.stdout:874: finishing write tid 3 to smithi138138488-47 2023-06-27T12:41:53.936 INFO:tasks.rados.rados.0.smithi138.stdout:874: finishing write tid 4 to smithi138138488-47 2023-06-27T12:41:53.936 INFO:tasks.rados.rados.0.smithi138.stdout:874: finishing write tid 5 to smithi138138488-47 2023-06-27T12:41:53.936 INFO:tasks.rados.rados.0.smithi138.stdout:874: oid 47 updating version 0 to 133 2023-06-27T12:41:53.937 INFO:tasks.rados.rados.0.smithi138.stdout:874: oid 47 updating version 133 to 136 2023-06-27T12:41:53.937 INFO:tasks.rados.rados.0.smithi138.stdout:874: oid 47 version 136 is already newer than 136 2023-06-27T12:41:53.937 INFO:tasks.rados.rados.0.smithi138.stdout:874: oid 47 version 136 is already newer than 136 2023-06-27T12:41:53.937 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 47 v 136 (ObjNum 422 snap 99 seq_num 422) dirty exists 2023-06-27T12:41:53.937 INFO:tasks.rados.rados.0.smithi138.stdout:874: left oid 47 (ObjNum 422 snap 99 seq_num 422) 2023-06-27T12:41:53.937 INFO:tasks.rados.rados.0.smithi138.stdout:875: finishing write tid 2 to smithi138138488-3 2023-06-27T12:41:53.937 INFO:tasks.rados.rados.0.smithi138.stdout:875: finishing write tid 3 to smithi138138488-3 2023-06-27T12:41:53.937 INFO:tasks.rados.rados.0.smithi138.stdout:875: finishing write tid 4 to smithi138138488-3 2023-06-27T12:41:53.938 INFO:tasks.rados.rados.0.smithi138.stdout:877: finishing write tid 1 to smithi138138488-40 2023-06-27T12:41:53.977 INFO:tasks.rados.rados.0.smithi138.stdout:875: finishing write tid 5 to smithi138138488-3 2023-06-27T12:41:53.977 INFO:tasks.rados.rados.0.smithi138.stdout:875: oid 3 updating version 0 to 127 2023-06-27T12:41:53.977 INFO:tasks.rados.rados.0.smithi138.stdout:875: oid 3 updating version 127 to 128 2023-06-27T12:41:53.977 INFO:tasks.rados.rados.0.smithi138.stdout:875: oid 3 version 128 is already newer than 128 2023-06-27T12:41:53.977 INFO:tasks.rados.rados.0.smithi138.stdout:875: oid 3 version 128 is already newer than 128 2023-06-27T12:41:53.978 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 128 (ObjNum 423 snap 99 seq_num 423) dirty exists 2023-06-27T12:41:53.978 INFO:tasks.rados.rados.0.smithi138.stdout:875: left oid 3 (ObjNum 423 snap 99 seq_num 423) 2023-06-27T12:41:53.984 INFO:tasks.rados.rados.0.smithi138.stdout:877: finishing write tid 2 to smithi138138488-40 2023-06-27T12:41:53.985 INFO:tasks.rados.rados.0.smithi138.stdout:877: finishing write tid 3 to smithi138138488-40 2023-06-27T12:41:53.986 INFO:tasks.rados.rados.0.smithi138.stdout:877: finishing write tid 4 to smithi138138488-40 2023-06-27T12:41:53.986 INFO:tasks.rados.rados.0.smithi138.stdout:877: oid 40 updating version 0 to 86 2023-06-27T12:41:53.987 INFO:tasks.rados.rados.0.smithi138.stdout:877: oid 40 version 86 is already newer than 86 2023-06-27T12:41:53.987 INFO:tasks.rados.rados.0.smithi138.stdout:877: oid 40 version 86 is already newer than 84 2023-06-27T12:41:53.987 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 86 (ObjNum 424 snap 99 seq_num 424) dirty exists 2023-06-27T12:41:53.987 INFO:tasks.rados.rados.0.smithi138.stdout:877: left oid 40 (ObjNum 424 snap 99 seq_num 424) 2023-06-27T12:41:53.995 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:54.122 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:54.117+0000 7ff1a6d22700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:54.124 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:54.121+0000 7ff1a6d22700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:54.169 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:54.171 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:54.172 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:54.255 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:54.256 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:54.256 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:41:54.384 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:54.377+0000 7f9a5fd5a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:54.386 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:54.383+0000 7f9a5fd5a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:54.497 INFO:tasks.rados.rados.0.smithi138.stdout:874: done (4 left) 2023-06-27T12:41:54.497 INFO:tasks.rados.rados.0.smithi138.stdout:875: done (3 left) 2023-06-27T12:41:54.498 INFO:tasks.rados.rados.0.smithi138.stdout:876: done (2 left) 2023-06-27T12:41:54.498 INFO:tasks.rados.rados.0.smithi138.stdout:877: done (1 left) 2023-06-27T12:41:54.498 INFO:tasks.rados.rados.0.smithi138.stdout:878: done (0 left) 2023-06-27T12:41:54.498 INFO:tasks.rados.rados.0.smithi138.stdout:879: write (excl) oid 48 current snap is 100 2023-06-27T12:41:54.498 INFO:tasks.rados.rados.0.smithi138.stdout:879: seq_num 425 ranges {742975=780240,1953553=427132,2971713=171262} 2023-06-27T12:41:54.505 INFO:tasks.rados.rados.0.smithi138.stdout:879: writing smithi138138488-48 from 742975 to 1523215 tid 1 2023-06-27T12:41:54.510 INFO:tasks.rados.rados.0.smithi138.stdout:879: writing smithi138138488-48 from 1953553 to 2380685 tid 2 2023-06-27T12:41:54.514 INFO:tasks.rados.rados.0.smithi138.stdout:879: writing smithi138138488-48 from 2971713 to 3142975 tid 3 2023-06-27T12:41:54.514 INFO:tasks.rados.rados.0.smithi138.stdout:880: delete oid 24 current snap is 100 2023-06-27T12:41:54.574 INFO:tasks.rados.rados.0.smithi138.stdout:880: done (1 left) 2023-06-27T12:41:54.574 INFO:tasks.rados.rados.0.smithi138.stdout:881: snap_remove snap 99 2023-06-27T12:41:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:41:54.599 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:41:54.611 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2023-06-27T12:41:54.492427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":87213305,"num_objects":212,"num_object_clones":170,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":1286,"num_read_kb":905001,"num_write":1082,"num_write_kb":312758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2205,"ondisk_log_size":2205,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":733848,"kb_used_data":609032,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376753512,"statfs":{"total":386547056640,"available":385795596288,"internally_reserved":0,"allocated":623648768,"data_stored":709333888,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18421000,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-5,"num_read":83,"num_read_kb":62746,"num_write":65,"num_write_kb":18345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023769"},"pg_stats":[{"pgid":"3.f","version":"205'10","reported_seq":510,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.494757+0000","last_change":"2023-06-27T12:41:52.502161+0000","last_active":"2023-06-27T12:41:53.494757+0000","last_peered":"2023-06-27T12:41:53.494757+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:53.494757+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2235397,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":8,"num_write_kb":2199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"220'136","reported_seq":694,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.877720+0000","last_change":"2023-06-27T12:41:52.568409+0000","last_active":"2023-06-27T12:41:53.877720+0000","last_peered":"2023-06-27T12:41:53.877720+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:53.877720+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9193627,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":97,"num_read_kb":107983,"num_write":91,"num_write_kb":26563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"219'169","reported_seq":690,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.711393+0000","last_change":"2023-06-27T12:41:52.651703+0000","last_active":"2023-06-27T12:41:53.711393+0000","last_peered":"2023-06-27T12:41:53.711393+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:53.711393+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933069,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":105,"num_read_kb":46650,"num_write":116,"num_write_kb":34162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"220'86","reported_seq":634,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.938474+0000","last_change":"2023-06-27T12:41:52.507177+0000","last_active":"2023-06-27T12:41:53.938474+0000","last_peered":"2023-06-27T12:41:53.938474+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:53.938474+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9529734,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":78,"num_read_kb":40228,"num_write":64,"num_write_kb":18271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"205'42","reported_seq":202,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.504674+0000","last_change":"2023-06-27T12:41:52.503070+0000","last_active":"2023-06-27T12:41:53.504674+0000","last_peered":"2023-06-27T12:41:53.504674+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:53.504674+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":1,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":24673,"num_write":9,"num_write_kb":2549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"219'277","reported_seq":697,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-06-27T12:41:53.501413+0000","last_change":"2023-06-27T12:41:52.501576+0000","last_active":"2023-06-27T12:41:53.501413+0000","last_peered":"2023-06-27T12:41:53.501413+0000","last_clean":"2023-06-27T12:41:53.501413+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:53.501413+0000","last_undegraded":"2023-06-27T12:41:53.501413+0000","last_fullsized":"2023-06-27T12:41:53.501413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10689483,"num_objects":31,"num_object_clones":25,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":232,"num_read_kb":149308,"num_write":161,"num_write_kb":45897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"205'143","reported_seq":560,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-06-27T12:41:53.500118+0000","last_change":"2023-06-27T12:41:52.506867+0000","last_active":"2023-06-27T12:41:53.500118+0000","last_peered":"2023-06-27T12:41:53.500118+0000","last_clean":"2023-06-27T12:41:53.500118+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:53.500118+0000","last_undegraded":"2023-06-27T12:41:53.500118+0000","last_fullsized":"2023-06-27T12:41:53.500118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2201783,"num_objects":18,"num_object_clones":15,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":94,"num_read_kb":64246,"num_write":80,"num_write_kb":23093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"199'60","reported_seq":587,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.528762+0000","last_change":"2023-06-27T12:41:52.611523+0000","last_active":"2023-06-27T12:41:53.528762+0000","last_peered":"2023-06-27T12:41:53.528762+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:53.528762+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916064,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":358,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-06-27T12:41:53.506515+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:53.506515+0000","last_peered":"2023-06-27T12:41:53.506515+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:53.506515+0000","last_undegraded":"2023-06-27T12:41:53.506515+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":509,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-06-27T12:41:53.534447+0000","last_change":"2023-06-27T12:41:52.652659+0000","last_active":"2023-06-27T12:41:53.534447+0000","last_peered":"2023-06-27T12:41:53.534447+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:53.534447+0000","last_undegraded":"2023-06-27T12:41:53.534447+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":335,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-06-27T12:41:53.497654+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:53.497654+0000","last_peered":"2023-06-27T12:41:53.497654+0000","last_clean":"2023-06-27T12:41:53.497654+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:53.497654+0000","last_undegraded":"2023-06-27T12:41:53.497654+0000","last_fullsized":"2023-06-27T12:41:53.497654+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"218'118","reported_seq":642,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.504246+0000","last_change":"2023-06-27T12:41:52.517475+0000","last_active":"2023-06-27T12:41:53.504246+0000","last_peered":"2023-06-27T12:41:53.504246+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:53.504246+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806835,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":58,"num_read_kb":56877,"num_write":73,"num_write_kb":19441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":346,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-06-27T12:41:53.536531+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:53.536531+0000","last_peered":"2023-06-27T12:41:53.536531+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:53.536531+0000","last_undegraded":"2023-06-27T12:41:53.536531+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"218'311","reported_seq":277,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.507956+0000","last_change":"2023-06-27T12:41:52.569173+0000","last_active":"2023-06-27T12:41:53.507956+0000","last_peered":"2023-06-27T12:41:53.507956+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:53.507956+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2790028,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":28,"num_read_kb":17194,"num_write":46,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":343,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-06-27T12:41:53.525367+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:53.525367+0000","last_peered":"2023-06-27T12:41:53.525367+0000","last_clean":"2023-06-27T12:41:53.525367+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:53.525367+0000","last_undegraded":"2023-06-27T12:41:53.525367+0000","last_fullsized":"2023-06-27T12:41:53.525367+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":87,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-06-27T12:41:53.500769+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:53.500769+0000","last_peered":"2023-06-27T12:41:53.500769+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:53.500769+0000","last_undegraded":"2023-06-27T12:41:53.500769+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"218'251","reported_seq":810,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.528244+0000","last_change":"2023-06-27T12:41:52.676575+0000","last_active":"2023-06-27T12:41:53.528244+0000","last_peered":"2023-06-27T12:41:53.528244+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:53.528244+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13849199,"num_objects":34,"num_object_clones":28,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":230,"num_read_kb":142998,"num_write":182,"num_write_kb":52663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":335,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-06-27T12:41:53.499922+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:53.499922+0000","last_peered":"2023-06-27T12:41:53.499922+0000","last_clean":"2023-06-27T12:41:53.499922+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:53.499922+0000","last_undegraded":"2023-06-27T12:41:53.499922+0000","last_fullsized":"2023-06-27T12:41:53.499922+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"220'143","reported_seq":249,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.798491+0000","last_change":"2023-06-27T12:41:52.651383+0000","last_active":"2023-06-27T12:41:53.798491+0000","last_peered":"2023-06-27T12:41:53.798491+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:53.798491+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117781,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":60,"num_read_kb":35684,"num_write":33,"num_write_kb":9435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-06-27T12:41:53.500586+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:53.500586+0000","last_peered":"2023-06-27T12:41:53.500586+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:53.500586+0000","last_undegraded":"2023-06-27T12:41:53.500586+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"220'128","reported_seq":673,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.912806+0000","last_change":"2023-06-27T12:41:52.515661+0000","last_active":"2023-06-27T12:41:53.912806+0000","last_peered":"2023-06-27T12:41:53.912806+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:53.912806+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11462572,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":84630,"num_write":91,"num_write_kb":27062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"218'93","reported_seq":244,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.530777+0000","last_change":"2023-06-27T12:41:52.686850+0000","last_active":"2023-06-27T12:41:53.530777+0000","last_peered":"2023-06-27T12:41:53.530777+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:53.530777+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2739607,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":50,"num_read_kb":26195,"num_write":26,"num_write_kb":7438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":358,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-06-27T12:41:53.509513+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:53.509513+0000","last_peered":"2023-06-27T12:41:53.509513+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:53.509513+0000","last_undegraded":"2023-06-27T12:41:53.509513+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":343,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-06-27T12:41:53.524286+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:53.524286+0000","last_peered":"2023-06-27T12:41:53.524286+0000","last_clean":"2023-06-27T12:41:53.524286+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:53.524286+0000","last_undegraded":"2023-06-27T12:41:53.524286+0000","last_fullsized":"2023-06-27T12:41:53.524286+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"220'141","reported_seq":648,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:53.732299+0000","last_change":"2023-06-27T12:41:52.510354+0000","last_active":"2023-06-27T12:41:53.732299+0000","last_peered":"2023-06-27T12:41:53.732299+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:53.732299+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5748126,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":66,"num_read_kb":65454,"num_write":65,"num_write_kb":18478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":14}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":87213305,"num_objects":212,"num_object_clones":170,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":1286,"num_read_kb":905001,"num_write":1082,"num_write_kb":312758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2108,"ondisk_log_size":2108,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673200,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":169508,"kb_used_data":138588,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94202332,"statfs":{"total":96636764160,"available":96463187968,"internally_reserved":0,"allocated":141914112,"data_stored":161512209,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":184392,"kb_used_data":152904,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94187448,"statfs":{"total":96636764160,"available":96447946752,"internally_reserved":0,"allocated":156573696,"data_stored":176880355,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673200,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":198668,"kb_used_data":167180,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94173172,"statfs":{"total":96636764160,"available":96433328128,"internally_reserved":0,"allocated":171192320,"data_stored":193901772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:41:54.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.b 3.3 3.1 3.4 3.7 2023-06-27T12:41:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:54.735+0000 7fa236a61700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:54.751 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:54.747+0000 7fa236a61700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:54.946 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b] on osd.0 to force-recovery; instructing pg(s) [3.1,3.7] on osd.1 to force-recovery; instructing pg(s) [3.3,3.4] on osd.2 to force-recovery; 2023-06-27T12:41:54.946 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:41:55.422 INFO:tasks.rados.rados.0.smithi138.stdout:881: done (1 left) 2023-06-27T12:41:55.422 INFO:tasks.rados.rados.0.smithi138.stdout:882: write (excl) oid 42 current snap is 100 2023-06-27T12:41:55.422 INFO:tasks.rados.rados.0.smithi138.stdout:879: finishing write tid 1 to smithi138138488-48 2023-06-27T12:41:55.422 INFO:tasks.rados.rados.0.smithi138.stdout:882: seq_num 427 ranges {776589=647091,2070517=430801,3026559=150030} 2023-06-27T12:41:55.429 INFO:tasks.rados.rados.0.smithi138.stdout:882: writing smithi138138488-42 from 776589 to 1423680 tid 1 2023-06-27T12:41:55.435 INFO:tasks.rados.rados.0.smithi138.stdout:882: writing smithi138138488-42 from 2070517 to 2501318 tid 2 2023-06-27T12:41:55.440 INFO:tasks.rados.rados.0.smithi138.stdout:882: writing smithi138138488-42 from 3026559 to 3176589 tid 3 2023-06-27T12:41:55.440 INFO:tasks.rados.rados.0.smithi138.stdout:883: snap_remove snap 98 2023-06-27T12:41:56.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:41:56.300 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:56.295+0000 7f1b78bff700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:56.303 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:56.299+0000 7f1b78bff700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:56.351 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:56.352 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:56.426 INFO:tasks.rados.rados.0.smithi138.stdout:883: done (2 left) 2023-06-27T12:41:56.426 INFO:tasks.rados.rados.0.smithi138.stdout:884: write (excl) oid 22 current snap is 100 2023-06-27T12:41:56.426 INFO:tasks.rados.rados.0.smithi138.stdout:879: finishing write tid 2 to smithi138138488-48 2023-06-27T12:41:56.426 INFO:tasks.rados.rados.0.smithi138.stdout:884: seq_num 428 ranges {793396=722340,2128999=690812} 2023-06-27T12:41:56.433 INFO:tasks.rados.rados.0.smithi138.stdout:884: writing smithi138138488-22 from 793396 to 1515736 tid 1 2023-06-27T12:41:56.442 INFO:tasks.rados.rados.0.smithi138.stdout:884: writing smithi138138488-22 from 2128999 to 2819811 tid 2 2023-06-27T12:41:56.443 INFO:tasks.rados.rados.0.smithi138.stdout:885: delete oid 16 current snap is 100 2023-06-27T12:41:56.443 INFO:tasks.rados.rados.0.smithi138.stdout:879: finishing write tid 3 to smithi138138488-48 2023-06-27T12:41:56.443 INFO:tasks.rados.rados.0.smithi138.stdout:879: finishing write tid 4 to smithi138138488-48 2023-06-27T12:41:56.443 INFO:tasks.rados.rados.0.smithi138.stdout:879: finishing write tid 5 to smithi138138488-48 2023-06-27T12:41:56.444 INFO:tasks.rados.rados.0.smithi138.stdout:879: oid 48 updating version 0 to 146 2023-06-27T12:41:56.444 INFO:tasks.rados.rados.0.smithi138.stdout:879: oid 48 version 146 is already newer than 146 2023-06-27T12:41:56.444 INFO:tasks.rados.rados.0.smithi138.stdout:879: oid 48 version 146 is already newer than 142 2023-06-27T12:41:56.444 INFO:tasks.rados.rados.0.smithi138.stdout:879: oid 48 version 146 is already newer than 146 2023-06-27T12:41:56.445 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 146 (ObjNum 425 snap 100 seq_num 425) dirty exists 2023-06-27T12:41:56.445 INFO:tasks.rados.rados.0.smithi138.stdout:879: left oid 48 (ObjNum 425 snap 100 seq_num 425) 2023-06-27T12:41:56.445 INFO:tasks.rados.rados.0.smithi138.stdout:882: finishing write tid 1 to smithi138138488-42 2023-06-27T12:41:56.446 INFO:tasks.rados.rados.0.smithi138.stdout:882: finishing write tid 2 to smithi138138488-42 2023-06-27T12:41:56.446 INFO:tasks.rados.rados.0.smithi138.stdout:882: finishing write tid 3 to smithi138138488-42 2023-06-27T12:41:56.446 INFO:tasks.rados.rados.0.smithi138.stdout:882: finishing write tid 4 to smithi138138488-42 2023-06-27T12:41:56.446 INFO:tasks.rados.rados.0.smithi138.stdout:882: finishing write tid 5 to smithi138138488-42 2023-06-27T12:41:56.447 INFO:tasks.rados.rados.0.smithi138.stdout:882: oid 42 updating version 0 to 91 2023-06-27T12:41:56.447 INFO:tasks.rados.rados.0.smithi138.stdout:882: oid 42 version 91 is already newer than 91 2023-06-27T12:41:56.447 INFO:tasks.rados.rados.0.smithi138.stdout:882: oid 42 version 91 is already newer than 87 2023-06-27T12:41:56.447 INFO:tasks.rados.rados.0.smithi138.stdout:882: oid 42 version 91 is already newer than 91 2023-06-27T12:41:56.448 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 91 (ObjNum 427 snap 100 seq_num 427) dirty exists 2023-06-27T12:41:56.448 INFO:tasks.rados.rados.0.smithi138.stdout:882: left oid 42 (ObjNum 427 snap 100 seq_num 427) 2023-06-27T12:41:56.477 INFO:tasks.rados.rados.0.smithi138.stdout:884: finishing write tid 1 to smithi138138488-22 2023-06-27T12:41:56.478 INFO:tasks.rados.rados.0.smithi138.stdout:879: done (3 left) 2023-06-27T12:41:56.478 INFO:tasks.rados.rados.0.smithi138.stdout:882: done (2 left) 2023-06-27T12:41:56.478 INFO:tasks.rados.rados.0.smithi138.stdout:885: done (1 left) 2023-06-27T12:41:56.479 INFO:tasks.rados.rados.0.smithi138.stdout:886: write (excl) oid 15 current snap is 100 2023-06-27T12:41:56.479 INFO:tasks.rados.rados.0.smithi138.stdout:886: seq_num 430 ranges {427010=472838,1329610=694481,2708828=518182} 2023-06-27T12:41:56.483 INFO:tasks.rados.rados.0.smithi138.stdout:886: writing smithi138138488-15 from 427010 to 899848 tid 1 2023-06-27T12:41:56.489 INFO:tasks.rados.rados.0.smithi138.stdout:886: writing smithi138138488-15 from 1329610 to 2024091 tid 2 2023-06-27T12:41:56.496 INFO:tasks.rados.rados.0.smithi138.stdout:886: writing smithi138138488-15 from 2708828 to 3227010 tid 3 2023-06-27T12:41:56.496 INFO:tasks.rados.rados.0.smithi138.stdout:887: write oid 33 current snap is 100 2023-06-27T12:41:56.496 INFO:tasks.rados.rados.0.smithi138.stdout:884: finishing write tid 2 to smithi138138488-22 2023-06-27T12:41:56.497 INFO:tasks.rados.rados.0.smithi138.stdout:887: seq_num 431 ranges {443817=548087,1788092=438139,3019898=223919} 2023-06-27T12:41:56.502 INFO:tasks.rados.rados.0.smithi138.stdout:887: writing smithi138138488-33 from 443817 to 991904 tid 1 2023-06-27T12:41:56.509 INFO:tasks.rados.rados.0.smithi138.stdout:887: writing smithi138138488-33 from 1788092 to 2226231 tid 2 2023-06-27T12:41:56.515 INFO:tasks.rados.rados.0.smithi138.stdout:887: writing smithi138138488-33 from 3019898 to 3243817 tid 3 2023-06-27T12:41:56.515 INFO:tasks.rados.rados.0.smithi138.stdout:888: delete oid 46 current snap is 100 2023-06-27T12:41:56.515 INFO:tasks.rados.rados.0.smithi138.stdout:884: finishing write tid 3 to smithi138138488-22 2023-06-27T12:41:56.515 INFO:tasks.rados.rados.0.smithi138.stdout:884: finishing write tid 4 to smithi138138488-22 2023-06-27T12:41:56.516 INFO:tasks.rados.rados.0.smithi138.stdout:884: oid 22 updating version 0 to 151 2023-06-27T12:41:56.516 INFO:tasks.rados.rados.0.smithi138.stdout:884: oid 22 version 151 is already newer than 151 2023-06-27T12:41:56.516 INFO:tasks.rados.rados.0.smithi138.stdout:884: oid 22 version 151 is already newer than 149 2023-06-27T12:41:56.516 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 151 (ObjNum 428 snap 100 seq_num 428) dirty exists 2023-06-27T12:41:56.516 INFO:tasks.rados.rados.0.smithi138.stdout:884: left oid 22 (ObjNum 428 snap 100 seq_num 428) 2023-06-27T12:41:56.554 INFO:tasks.rados.rados.0.smithi138.stdout:886: finishing write tid 1 to smithi138138488-15 2023-06-27T12:41:56.562 INFO:tasks.rados.rados.0.smithi138.stdout:884: done (3 left) 2023-06-27T12:41:56.563 INFO:tasks.rados.rados.0.smithi138.stdout:888: done (2 left) 2023-06-27T12:41:56.563 INFO:tasks.rados.rados.0.smithi138.stdout:889: write oid 38 current snap is 100 2023-06-27T12:41:56.563 INFO:tasks.rados.rados.0.smithi138.stdout:889: seq_num 433 ranges {477431=698585,1905056=441808,3074744=202687} 2023-06-27T12:41:56.569 INFO:tasks.rados.rados.0.smithi138.stdout:889: writing smithi138138488-38 from 477431 to 1176016 tid 1 2023-06-27T12:41:56.576 INFO:tasks.rados.rados.0.smithi138.stdout:889: writing smithi138138488-38 from 1905056 to 2346864 tid 2 2023-06-27T12:41:56.581 INFO:tasks.rados.rados.0.smithi138.stdout:889: writing smithi138138488-38 from 3074744 to 3277431 tid 3 2023-06-27T12:41:56.582 INFO:tasks.rados.rados.0.smithi138.stdout:890: write (excl) oid 21 current snap is 100 2023-06-27T12:41:56.582 INFO:tasks.rados.rados.0.smithi138.stdout:890: seq_num 434 ranges {494238=490187,1563538=701819,2818520=475718} 2023-06-27T12:41:56.588 INFO:tasks.rados.rados.0.smithi138.stdout:890: writing smithi138138488-21 from 494238 to 984425 tid 1 2023-06-27T12:41:56.595 INFO:tasks.rados.rados.0.smithi138.stdout:890: writing smithi138138488-21 from 1563538 to 2265357 tid 2 2023-06-27T12:41:56.603 INFO:tasks.rados.rados.0.smithi138.stdout:890: writing smithi138138488-21 from 2818520 to 3294238 tid 3 2023-06-27T12:41:56.603 INFO:tasks.rados.rados.0.smithi138.stdout:891: read oid 13 snap -1 2023-06-27T12:41:56.603 INFO:tasks.rados.rados.0.smithi138.stdout:891: expect deleted 2023-06-27T12:41:56.604 INFO:tasks.rados.rados.0.smithi138.stdout:892: read oid 36 snap -1 2023-06-27T12:41:56.604 INFO:tasks.rados.rados.0.smithi138.stdout:892: expect (ObjNum 263 snap 70 seq_num 263) 2023-06-27T12:41:56.604 INFO:tasks.rados.rados.0.smithi138.stdout:893: write oid 6 current snap is 100 2023-06-27T12:41:56.604 INFO:tasks.rados.rados.0.smithi138.stdout:893: seq_num 435 ranges {511045=565436,1622020=445477,2729590=556855} 2023-06-27T12:41:56.608 INFO:tasks.rados.rados.0.smithi138.stdout:893: writing smithi138138488-6 from 511045 to 1076481 tid 1 2023-06-27T12:41:56.614 INFO:tasks.rados.rados.0.smithi138.stdout:893: writing smithi138138488-6 from 1622020 to 2067497 tid 2 2023-06-27T12:41:56.621 INFO:tasks.rados.rados.0.smithi138.stdout:893: writing smithi138138488-6 from 2729590 to 3286445 tid 3 2023-06-27T12:41:56.621 INFO:tasks.rados.rados.0.smithi138.stdout:894: read oid 23 snap -1 2023-06-27T12:41:56.622 INFO:tasks.rados.rados.0.smithi138.stdout:894: expect deleted 2023-06-27T12:41:56.622 INFO:tasks.rados.rados.0.smithi138.stdout:895: snap_create 2023-06-27T12:41:56.672 INFO:tasks.rados.rados.0.smithi138.stdout:887: finishing write tid 1 to smithi138138488-33 2023-06-27T12:41:56.713 INFO:tasks.rados.rados.0.smithi138.stdout:886: finishing write tid 2 to smithi138138488-15 2023-06-27T12:41:56.714 INFO:tasks.rados.rados.0.smithi138.stdout:886: finishing write tid 3 to smithi138138488-15 2023-06-27T12:41:56.714 INFO:tasks.rados.rados.0.smithi138.stdout:886: finishing write tid 4 to smithi138138488-15 2023-06-27T12:41:56.714 INFO:tasks.rados.rados.0.smithi138.stdout:889: finishing write tid 1 to smithi138138488-38 2023-06-27T12:41:56.714 INFO:tasks.rados.rados.0.smithi138.stdout:886: finishing write tid 5 to smithi138138488-15 2023-06-27T12:41:56.714 INFO:tasks.rados.rados.0.smithi138.stdout:886: oid 15 updating version 0 to 46 2023-06-27T12:41:56.715 INFO:tasks.rados.rados.0.smithi138.stdout:886: oid 15 version 46 is already newer than 46 2023-06-27T12:41:56.715 INFO:tasks.rados.rados.0.smithi138.stdout:886: oid 15 version 46 is already newer than 43 2023-06-27T12:41:56.715 INFO:tasks.rados.rados.0.smithi138.stdout:886: oid 15 version 46 is already newer than 46 2023-06-27T12:41:56.715 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 46 (ObjNum 430 snap 100 seq_num 430) dirty exists 2023-06-27T12:41:56.715 INFO:tasks.rados.rados.0.smithi138.stdout:886: left oid 15 (ObjNum 430 snap 100 seq_num 430) 2023-06-27T12:41:56.715 INFO:tasks.rados.rados.0.smithi138.stdout:887: finishing write tid 2 to smithi138138488-33 2023-06-27T12:41:56.715 INFO:tasks.rados.rados.0.smithi138.stdout:889: finishing write tid 2 to smithi138138488-38 2023-06-27T12:41:56.715 INFO:tasks.rados.rados.0.smithi138.stdout:887: finishing write tid 3 to smithi138138488-33 2023-06-27T12:41:56.716 INFO:tasks.rados.rados.0.smithi138.stdout:892: expect (ObjNum 263 snap 70 seq_num 263) 2023-06-27T12:41:56.846 INFO:tasks.rados.rados.0.smithi138.stdout:889: finishing write tid 3 to smithi138138488-38 2023-06-27T12:41:56.847 INFO:tasks.rados.rados.0.smithi138.stdout:890: finishing write tid 1 to smithi138138488-21 2023-06-27T12:41:56.847 INFO:tasks.rados.rados.0.smithi138.stdout:887: finishing write tid 4 to smithi138138488-33 2023-06-27T12:41:56.847 INFO:tasks.rados.rados.0.smithi138.stdout:893: finishing write tid 1 to smithi138138488-6 2023-06-27T12:41:56.847 INFO:tasks.rados.rados.0.smithi138.stdout:887: finishing write tid 5 to smithi138138488-33 2023-06-27T12:41:56.847 INFO:tasks.rados.rados.0.smithi138.stdout:887: oid 33 updating version 0 to 95 2023-06-27T12:41:56.847 INFO:tasks.rados.rados.0.smithi138.stdout:887: oid 33 version 95 is already newer than 95 2023-06-27T12:41:56.848 INFO:tasks.rados.rados.0.smithi138.stdout:887: oid 33 version 95 is already newer than 95 2023-06-27T12:41:56.848 INFO:tasks.rados.rados.0.smithi138.stdout:887: oid 33 version 95 is already newer than 95 2023-06-27T12:41:56.848 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 95 (ObjNum 431 snap 100 seq_num 431) dirty exists 2023-06-27T12:41:56.848 INFO:tasks.rados.rados.0.smithi138.stdout:887: left oid 33 (ObjNum 431 snap 100 seq_num 431) 2023-06-27T12:41:56.848 INFO:tasks.rados.rados.0.smithi138.stdout:889: finishing write tid 4 to smithi138138488-38 2023-06-27T12:41:56.848 INFO:tasks.rados.rados.0.smithi138.stdout:889: finishing write tid 5 to smithi138138488-38 2023-06-27T12:41:56.848 INFO:tasks.rados.rados.0.smithi138.stdout:889: oid 38 updating version 0 to 14 2023-06-27T12:41:56.849 INFO:tasks.rados.rados.0.smithi138.stdout:889: oid 38 version 14 is already newer than 14 2023-06-27T12:41:56.849 INFO:tasks.rados.rados.0.smithi138.stdout:889: oid 38 version 14 is already newer than 14 2023-06-27T12:41:56.849 INFO:tasks.rados.rados.0.smithi138.stdout:889: oid 38 version 14 is already newer than 14 2023-06-27T12:41:56.849 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 14 (ObjNum 433 snap 100 seq_num 433) dirty exists 2023-06-27T12:41:56.849 INFO:tasks.rados.rados.0.smithi138.stdout:889: left oid 38 (ObjNum 433 snap 100 seq_num 433) 2023-06-27T12:41:56.849 INFO:tasks.rados.rados.0.smithi138.stdout:893: finishing write tid 2 to smithi138138488-6 2023-06-27T12:41:56.849 INFO:tasks.rados.rados.0.smithi138.stdout:893: finishing write tid 3 to smithi138138488-6 2023-06-27T12:41:56.849 INFO:tasks.rados.rados.0.smithi138.stdout:893: finishing write tid 4 to smithi138138488-6 2023-06-27T12:41:56.850 INFO:tasks.rados.rados.0.smithi138.stdout:893: finishing write tid 5 to smithi138138488-6 2023-06-27T12:41:56.850 INFO:tasks.rados.rados.0.smithi138.stdout:893: oid 6 updating version 0 to 123 2023-06-27T12:41:56.850 INFO:tasks.rados.rados.0.smithi138.stdout:893: oid 6 version 123 is already newer than 123 2023-06-27T12:41:56.850 INFO:tasks.rados.rados.0.smithi138.stdout:893: oid 6 version 123 is already newer than 123 2023-06-27T12:41:56.850 INFO:tasks.rados.rados.0.smithi138.stdout:893: oid 6 version 123 is already newer than 123 2023-06-27T12:41:56.850 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 123 (ObjNum 435 snap 100 seq_num 435) dirty exists 2023-06-27T12:41:56.851 INFO:tasks.rados.rados.0.smithi138.stdout:893: left oid 6 (ObjNum 435 snap 100 seq_num 435) 2023-06-27T12:41:56.851 INFO:tasks.rados.rados.0.smithi138.stdout:890: finishing write tid 2 to smithi138138488-21 2023-06-27T12:41:56.851 INFO:tasks.rados.rados.0.smithi138.stdout:890: finishing write tid 3 to smithi138138488-21 2023-06-27T12:41:56.851 INFO:tasks.rados.rados.0.smithi138.stdout:890: finishing write tid 4 to smithi138138488-21 2023-06-27T12:41:56.851 INFO:tasks.rados.rados.0.smithi138.stdout:890: finishing write tid 5 to smithi138138488-21 2023-06-27T12:41:56.851 INFO:tasks.rados.rados.0.smithi138.stdout:890: oid 21 updating version 0 to 257 2023-06-27T12:41:56.851 INFO:tasks.rados.rados.0.smithi138.stdout:890: oid 21 version 257 is already newer than 254 2023-06-27T12:41:56.851 INFO:tasks.rados.rados.0.smithi138.stdout:890: oid 21 version 257 is already newer than 257 2023-06-27T12:41:56.852 INFO:tasks.rados.rados.0.smithi138.stdout:890: oid 21 version 257 is already newer than 257 2023-06-27T12:41:56.852 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 21 v 257 (ObjNum 434 snap 100 seq_num 434) dirty exists 2023-06-27T12:41:56.852 INFO:tasks.rados.rados.0.smithi138.stdout:890: left oid 21 (ObjNum 434 snap 100 seq_num 434) 2023-06-27T12:41:57.503 INFO:tasks.rados.rados.0.smithi138.stdout:886: done (8 left) 2023-06-27T12:41:57.504 INFO:tasks.rados.rados.0.smithi138.stdout:887: done (7 left) 2023-06-27T12:41:57.504 INFO:tasks.rados.rados.0.smithi138.stdout:889: done (6 left) 2023-06-27T12:41:57.504 INFO:tasks.rados.rados.0.smithi138.stdout:890: done (5 left) 2023-06-27T12:41:57.504 INFO:tasks.rados.rados.0.smithi138.stdout:891: done (4 left) 2023-06-27T12:41:57.504 INFO:tasks.rados.rados.0.smithi138.stdout:892: done (3 left) 2023-06-27T12:41:57.505 INFO:tasks.rados.rados.0.smithi138.stdout:893: done (2 left) 2023-06-27T12:41:57.505 INFO:tasks.rados.rados.0.smithi138.stdout:894: done (1 left) 2023-06-27T12:41:57.505 INFO:tasks.rados.rados.0.smithi138.stdout:895: done (0 left) 2023-06-27T12:41:57.505 INFO:tasks.rados.rados.0.smithi138.stdout:896: snap_create 2023-06-27T12:41:57.868 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~83s 2023-06-27T12:41:58.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:41:58.493 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:58.488+0000 7fe366c8d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:58.496 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:41:58.492+0000 7fe366c8d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:41:58.506 INFO:tasks.rados.rados.0.smithi138.stdout:896: done (0 left) 2023-06-27T12:41:58.506 INFO:tasks.rados.rados.0.smithi138.stdout:897: delete oid 13 current snap is 102 2023-06-27T12:41:58.526 INFO:tasks.rados.rados.0.smithi138.stdout:897: done (0 left) 2023-06-27T12:41:58.526 INFO:tasks.rados.rados.0.smithi138.stdout:898: delete oid 45 current snap is 102 2023-06-27T12:41:58.541 INFO:tasks.rados.rados.0.smithi138.stdout:898: done (0 left) 2023-06-27T12:41:58.541 INFO:tasks.rados.rados.0.smithi138.stdout:899: delete oid 48 current snap is 102 2023-06-27T12:41:58.551 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:41:58.554 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:41:58.554 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:41:58.592 INFO:tasks.rados.rados.0.smithi138.stdout:899: done (0 left) 2023-06-27T12:41:58.592 INFO:tasks.rados.rados.0.smithi138.stdout:900: read oid 11 snap -1 2023-06-27T12:41:58.593 INFO:tasks.rados.rados.0.smithi138.stdout:900: expect (ObjNum 389 snap 95 seq_num 389) 2023-06-27T12:41:58.593 INFO:tasks.rados.rados.0.smithi138.stdout:901: read oid 46 snap -1 2023-06-27T12:41:58.593 INFO:tasks.rados.rados.0.smithi138.stdout:901: expect deleted 2023-06-27T12:41:58.593 INFO:tasks.rados.rados.0.smithi138.stdout:902: read oid 16 snap -1 2023-06-27T12:41:58.593 INFO:tasks.rados.rados.0.smithi138.stdout:902: expect deleted 2023-06-27T12:41:58.593 INFO:tasks.rados.rados.0.smithi138.stdout:903: snap_create 2023-06-27T12:41:58.637 INFO:tasks.rados.rados.0.smithi138.stdout:900: expect (ObjNum 389 snap 95 seq_num 389) 2023-06-27T12:41:59.519 INFO:tasks.rados.rados.0.smithi138.stdout:900: done (3 left) 2023-06-27T12:41:59.519 INFO:tasks.rados.rados.0.smithi138.stdout:901: done (2 left) 2023-06-27T12:41:59.519 INFO:tasks.rados.rados.0.smithi138.stdout:902: done (1 left) 2023-06-27T12:41:59.520 INFO:tasks.rados.rados.0.smithi138.stdout:903: done (0 left) 2023-06-27T12:41:59.520 INFO:tasks.rados.rados.0.smithi138.stdout:904: delete oid 49 current snap is 103 2023-06-27T12:41:59.596 INFO:tasks.rados.rados.0.smithi138.stdout:904: done (0 left) 2023-06-27T12:41:59.596 INFO:tasks.rados.rados.0.smithi138.stdout:905: write oid 8 current snap is 103 2023-06-27T12:41:59.596 INFO:tasks.rados.rados.0.smithi138.stdout:905: seq_num 440 ranges {595080=541681,1798077=596473,3150352=244728} 2023-06-27T12:41:59.602 INFO:tasks.rados.rados.0.smithi138.stdout:905: writing smithi138138488-8 from 595080 to 1136761 tid 1 2023-06-27T12:41:59.610 INFO:tasks.rados.rados.0.smithi138.stdout:905: writing smithi138138488-8 from 1798077 to 2394550 tid 2 2023-06-27T12:41:59.617 INFO:tasks.rados.rados.0.smithi138.stdout:905: writing smithi138138488-8 from 3150352 to 3395080 tid 3 2023-06-27T12:41:59.617 INFO:tasks.rados.rados.0.smithi138.stdout:906: write (excl) oid 46 current snap is 103 2023-06-27T12:41:59.617 INFO:tasks.rados.rados.0.smithi138.stdout:906: seq_num 441 ranges {611887=733283,1972912=456484,2894128=517759} 2023-06-27T12:41:59.624 INFO:tasks.rados.rados.0.smithi138.stdout:906: writing smithi138138488-46 from 611887 to 1345170 tid 1 2023-06-27T12:41:59.630 INFO:tasks.rados.rados.0.smithi138.stdout:906: writing smithi138138488-46 from 1972912 to 2429396 tid 2 2023-06-27T12:41:59.636 INFO:tasks.rados.rados.0.smithi138.stdout:906: writing smithi138138488-46 from 2894128 to 3411887 tid 3 2023-06-27T12:41:59.636 INFO:tasks.rados.rados.0.smithi138.stdout:907: snap_remove snap 102 2023-06-27T12:41:59.958 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:41:59.959 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:41:59.959 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:00.089 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:00.082+0000 7f8b30172700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:00.092 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:00.088+0000 7f8b30172700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:00.285 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:00.286 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:00.298 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2023-06-27T12:41:59.955893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":85666995,"num_objects":217,"num_object_clones":175,"num_object_copies":651,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1316,"num_read_kb":913577,"num_write":1121,"num_write_kb":323994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2247,"ondisk_log_size":2247,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":742160,"kb_used_data":618480,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376745200,"statfs":{"total":386547056640,"available":385787084800,"internally_reserved":0,"allocated":633323520,"data_stored":715879879,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1546310,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":8576,"num_write":39,"num_write_kb":11236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001020"},"pg_stats":[{"pgid":"3.f","version":"223'14","reported_seq":527,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.610247+0000","last_change":"2023-06-27T12:41:56.454177+0000","last_active":"2023-06-27T12:41:58.610247+0000","last_peered":"2023-06-27T12:41:58.610247+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:41:58.610247+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3277431,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":13,"num_write_kb":3512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"220'136","reported_seq":704,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.519438+0000","last_change":"2023-06-27T12:41:56.462544+0000","last_active":"2023-06-27T12:41:58.519438+0000","last_peered":"2023-06-27T12:41:58.519438+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:41:58.519438+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9193627,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":97,"num_read_kb":107983,"num_write":91,"num_write_kb":26563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"219'169","reported_seq":704,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.631760+0000","last_change":"2023-06-27T12:41:56.608491+0000","last_active":"2023-06-27T12:41:58.631760+0000","last_peered":"2023-06-27T12:41:58.631760+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:41:58.631760+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933069,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":119,"num_read_kb":55218,"num_write":116,"num_write_kb":34162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"223'95","reported_seq":657,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.614036+0000","last_change":"2023-06-27T12:41:56.482249+0000","last_active":"2023-06-27T12:41:58.614036+0000","last_peered":"2023-06-27T12:41:58.614036+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:41:58.614036+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9546574,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":81,"num_read_kb":40230,"num_write":74,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"223'46","reported_seq":219,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.620862+0000","last_change":"2023-06-27T12:41:56.454609+0000","last_active":"2023-06-27T12:41:58.620862+0000","last_peered":"2023-06-27T12:41:58.620862+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:41:58.620862+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1042034,"num_objects":1,"num_object_clones":1,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":24674,"num_write":14,"num_write_kb":4197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"221'279","reported_seq":712,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-06-27T12:41:58.615315+0000","last_change":"2023-06-27T12:41:56.456395+0000","last_active":"2023-06-27T12:41:58.615315+0000","last_peered":"2023-06-27T12:41:58.615315+0000","last_clean":"2023-06-27T12:41:58.615315+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:41:58.615315+0000","last_undegraded":"2023-06-27T12:41:58.615315+0000","last_fullsized":"2023-06-27T12:41:58.615315+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8319630,"num_objects":32,"num_object_clones":26,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":234,"num_read_kb":149308,"num_write":161,"num_write_kb":45897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"220'145","reported_seq":572,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-06-27T12:41:58.607583+0000","last_change":"2023-06-27T12:41:56.533820+0000","last_active":"2023-06-27T12:41:58.607583+0000","last_peered":"2023-06-27T12:41:58.607583+0000","last_clean":"2023-06-27T12:41:58.607583+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:41:58.607583+0000","last_undegraded":"2023-06-27T12:41:58.607583+0000","last_fullsized":"2023-06-27T12:41:58.607583+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2201783,"num_objects":18,"num_object_clones":15,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":94,"num_read_kb":64246,"num_write":80,"num_write_kb":23093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"199'60","reported_seq":595,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.619029+0000","last_change":"2023-06-27T12:41:56.558419+0000","last_active":"2023-06-27T12:41:58.619029+0000","last_peered":"2023-06-27T12:41:58.619029+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:41:58.619029+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916064,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":366,"reported_epoch":225,"state":"active+undersized","last_fresh":"2023-06-27T12:41:58.514850+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:41:58.514850+0000","last_peered":"2023-06-27T12:41:58.514850+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:41:58.514850+0000","last_undegraded":"2023-06-27T12:41:58.514850+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":517,"reported_epoch":225,"state":"active+undersized","last_fresh":"2023-06-27T12:41:58.611676+0000","last_change":"2023-06-27T12:41:56.613630+0000","last_active":"2023-06-27T12:41:58.611676+0000","last_peered":"2023-06-27T12:41:58.611676+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:41:58.611676+0000","last_undegraded":"2023-06-27T12:41:58.611676+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":345,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-06-27T12:41:58.615935+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:41:58.615935+0000","last_peered":"2023-06-27T12:41:58.615935+0000","last_clean":"2023-06-27T12:41:58.615935+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:41:58.615935+0000","last_undegraded":"2023-06-27T12:41:58.615935+0000","last_fullsized":"2023-06-27T12:41:58.615935+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"223'123","reported_seq":658,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.516006+0000","last_change":"2023-06-27T12:41:56.466821+0000","last_active":"2023-06-27T12:41:58.516006+0000","last_peered":"2023-06-27T12:41:58.516006+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:41:58.516006+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3311045,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":59,"num_read_kb":56878,"num_write":78,"num_write_kb":20974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":352,"reported_epoch":225,"state":"active+undersized","last_fresh":"2023-06-27T12:41:58.618782+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:41:58.618782+0000","last_peered":"2023-06-27T12:41:58.618782+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:41:58.618782+0000","last_undegraded":"2023-06-27T12:41:58.618782+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"218'311","reported_seq":289,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.522598+0000","last_change":"2023-06-27T12:41:56.465450+0000","last_active":"2023-06-27T12:41:58.522598+0000","last_peered":"2023-06-27T12:41:58.522598+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:41:58.522598+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":28,"num_read_kb":17194,"num_write":46,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":349,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-06-27T12:41:58.613517+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:41:58.613517+0000","last_peered":"2023-06-27T12:41:58.613517+0000","last_clean":"2023-06-27T12:41:58.613517+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:41:58.613517+0000","last_undegraded":"2023-06-27T12:41:58.613517+0000","last_fullsized":"2023-06-27T12:41:58.613517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":97,"reported_epoch":225,"state":"active+undersized","last_fresh":"2023-06-27T12:41:58.609880+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:41:58.609880+0000","last_peered":"2023-06-27T12:41:58.609880+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:41:58.609880+0000","last_undegraded":"2023-06-27T12:41:58.609880+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"223'257","reported_seq":823,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.609302+0000","last_change":"2023-06-27T12:41:56.549158+0000","last_active":"2023-06-27T12:41:58.609302+0000","last_peered":"2023-06-27T12:41:58.609302+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:41:58.609302+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14689582,"num_objects":34,"num_object_clones":28,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":232,"num_read_kb":142999,"num_write":187,"num_write_kb":54293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":345,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-06-27T12:41:58.614647+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:41:58.614647+0000","last_peered":"2023-06-27T12:41:58.614647+0000","last_clean":"2023-06-27T12:41:58.614647+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:41:58.614647+0000","last_undegraded":"2023-06-27T12:41:58.614647+0000","last_fullsized":"2023-06-27T12:41:58.614647+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"220'143","reported_seq":257,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.626080+0000","last_change":"2023-06-27T12:41:57.821180+0000","last_active":"2023-06-27T12:41:58.626080+0000","last_peered":"2023-06-27T12:41:58.626080+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:41:58.626080+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117781,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":60,"num_read_kb":35684,"num_write":33,"num_write_kb":9435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":109,"reported_epoch":225,"state":"active+undersized","last_fresh":"2023-06-27T12:41:58.511897+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:41:58.511897+0000","last_peered":"2023-06-27T12:41:58.511897+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:41:58.511897+0000","last_undegraded":"2023-06-27T12:41:58.511897+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"220'128","reported_seq":683,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.513380+0000","last_change":"2023-06-27T12:41:56.450350+0000","last_active":"2023-06-27T12:41:58.513380+0000","last_peered":"2023-06-27T12:41:58.513380+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:41:58.513380+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11462572,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":84630,"num_write":91,"num_write_kb":27062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"218'93","reported_seq":252,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.612344+0000","last_change":"2023-06-27T12:41:56.562585+0000","last_active":"2023-06-27T12:41:58.612344+0000","last_peered":"2023-06-27T12:41:58.612344+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:41:58.612344+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2739607,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":50,"num_read_kb":26195,"num_write":26,"num_write_kb":7438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":366,"reported_epoch":225,"state":"active+undersized","last_fresh":"2023-06-27T12:41:58.518963+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:41:58.518963+0000","last_peered":"2023-06-27T12:41:58.518963+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:41:58.518963+0000","last_undegraded":"2023-06-27T12:41:58.518963+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":349,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-06-27T12:41:58.616990+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:41:58.616990+0000","last_peered":"2023-06-27T12:41:58.616990+0000","last_clean":"2023-06-27T12:41:58.616990+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:41:58.616990+0000","last_undegraded":"2023-06-27T12:41:58.616990+0000","last_fullsized":"2023-06-27T12:41:58.616990+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"223'151","reported_seq":668,"reported_epoch":225,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:41:58.517644+0000","last_change":"2023-06-27T12:41:56.480425+0000","last_active":"2023-06-27T12:41:58.517644+0000","last_peered":"2023-06-27T12:41:58.517644+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:41:58.517644+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5916196,"num_objects":16,"num_object_clones":13,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":71,"num_read_kb":65456,"num_write":74,"num_write_kb":21207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":16}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":85666995,"num_objects":217,"num_object_clones":175,"num_object_copies":651,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1316,"num_read_kb":913577,"num_write":1121,"num_write_kb":323994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2150,"ondisk_log_size":2150,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":174284,"kb_used_data":143364,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94197556,"statfs":{"total":96636764160,"available":96458297344,"internally_reserved":0,"allocated":146804736,"data_stored":165783587,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":187604,"kb_used_data":156684,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94184236,"statfs":{"total":96636764160,"available":96444657664,"internally_reserved":0,"allocated":160444416,"data_stored":179708926,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":198992,"kb_used_data":168072,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94172848,"statfs":{"total":96636764160,"available":96432996352,"internally_reserved":0,"allocated":172105728,"data_stored":193347814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:00.300 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.d 3.8 3.3 3.0 3.7 3.6 2023-06-27T12:42:00.436 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:00.420+0000 7f5b5827c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:00.439 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:00.435+0000 7f5b5827c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:00.530 INFO:tasks.rados.rados.0.smithi138.stdout:907: done (2 left) 2023-06-27T12:42:00.531 INFO:tasks.rados.rados.0.smithi138.stdout:908: snap_remove snap 101 2023-06-27T12:42:00.555 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:00.639 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.f] on osd.0 to force-recovery; instructing pg(s) [3.7,3.8,3.d] on osd.1 to force-recovery; instructing pg(s) [3.0,3.3,3.6,3.e] on osd.2 to force-recovery; 2023-06-27T12:42:00.640 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:42:00.683 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:00.678+0000 7fdacb960700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:00.686 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:00.682+0000 7fdacb960700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:00.732 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:00.735 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:00.736 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:01.530 INFO:tasks.rados.rados.0.smithi138.stdout:908: done (2 left) 2023-06-27T12:42:01.531 INFO:tasks.rados.rados.0.smithi138.stdout:909: read oid 7 snap -1 2023-06-27T12:42:01.531 INFO:tasks.rados.rados.0.smithi138.stdout:909: expect (ObjNum 369 snap 91 seq_num 369) 2023-06-27T12:42:01.531 INFO:tasks.rados.rados.0.smithi138.stdout:910: snap_remove snap 100 2023-06-27T12:42:02.534 INFO:tasks.rados.rados.0.smithi138.stdout:910: done (3 left) 2023-06-27T12:42:02.535 INFO:tasks.rados.rados.0.smithi138.stdout:911: write oid 28 current snap is 103 2023-06-27T12:42:02.535 INFO:tasks.rados.rados.0.smithi138.stdout:905: finishing write tid 1 to smithi138138488-8 2023-06-27T12:42:02.535 INFO:tasks.rados.rados.0.smithi138.stdout:911: seq_num 442 ranges {628694=408532,1515041=600142,2805198=552112} 2023-06-27T12:42:02.540 INFO:tasks.rados.rados.0.smithi138.stdout:911: writing smithi138138488-28 from 628694 to 1037226 tid 1 2023-06-27T12:42:02.546 INFO:tasks.rados.rados.0.smithi138.stdout:911: writing smithi138138488-28 from 1515041 to 2115183 tid 2 2023-06-27T12:42:02.554 INFO:tasks.rados.rados.0.smithi138.stdout:911: writing smithi138138488-28 from 2805198 to 3357310 tid 3 2023-06-27T12:42:02.554 INFO:tasks.rados.rados.0.smithi138.stdout:912: read oid 23 snap -1 2023-06-27T12:42:02.554 INFO:tasks.rados.rados.0.smithi138.stdout:912: expect deleted 2023-06-27T12:42:02.554 INFO:tasks.rados.rados.0.smithi138.stdout:913: delete oid 34 current snap is 103 2023-06-27T12:42:02.554 INFO:tasks.rados.rados.0.smithi138.stdout:905: finishing write tid 2 to smithi138138488-8 2023-06-27T12:42:02.555 INFO:tasks.rados.rados.0.smithi138.stdout:905: finishing write tid 3 to smithi138138488-8 2023-06-27T12:42:02.555 INFO:tasks.rados.rados.0.smithi138.stdout:905: finishing write tid 4 to smithi138138488-8 2023-06-27T12:42:02.555 INFO:tasks.rados.rados.0.smithi138.stdout:905: finishing write tid 5 to smithi138138488-8 2023-06-27T12:42:02.555 INFO:tasks.rados.rados.0.smithi138.stdout:905: oid 8 updating version 0 to 50 2023-06-27T12:42:02.555 INFO:tasks.rados.rados.0.smithi138.stdout:905: oid 8 version 50 is already newer than 50 2023-06-27T12:42:02.555 INFO:tasks.rados.rados.0.smithi138.stdout:905: oid 8 version 50 is already newer than 48 2023-06-27T12:42:02.555 INFO:tasks.rados.rados.0.smithi138.stdout:905: oid 8 version 50 is already newer than 50 2023-06-27T12:42:02.556 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 8 v 50 (ObjNum 440 snap 103 seq_num 440) dirty exists 2023-06-27T12:42:02.556 INFO:tasks.rados.rados.0.smithi138.stdout:905: left oid 8 (ObjNum 440 snap 103 seq_num 440) 2023-06-27T12:42:02.556 INFO:tasks.rados.rados.0.smithi138.stdout:906: finishing write tid 1 to smithi138138488-46 2023-06-27T12:42:02.556 INFO:tasks.rados.rados.0.smithi138.stdout:906: finishing write tid 2 to smithi138138488-46 2023-06-27T12:42:02.556 INFO:tasks.rados.rados.0.smithi138.stdout:906: finishing write tid 3 to smithi138138488-46 2023-06-27T12:42:02.556 INFO:tasks.rados.rados.0.smithi138.stdout:906: finishing write tid 4 to smithi138138488-46 2023-06-27T12:42:02.556 INFO:tasks.rados.rados.0.smithi138.stdout:906: finishing write tid 5 to smithi138138488-46 2023-06-27T12:42:02.557 INFO:tasks.rados.rados.0.smithi138.stdout:906: oid 46 updating version 0 to 124 2023-06-27T12:42:02.557 INFO:tasks.rados.rados.0.smithi138.stdout:906: oid 46 updating version 124 to 127 2023-06-27T12:42:02.557 INFO:tasks.rados.rados.0.smithi138.stdout:906: oid 46 version 127 is already newer than 127 2023-06-27T12:42:02.557 INFO:tasks.rados.rados.0.smithi138.stdout:906: oid 46 version 127 is already newer than 127 2023-06-27T12:42:02.557 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 127 (ObjNum 441 snap 103 seq_num 441) dirty exists 2023-06-27T12:42:02.557 INFO:tasks.rados.rados.0.smithi138.stdout:906: left oid 46 (ObjNum 441 snap 103 seq_num 441) 2023-06-27T12:42:02.557 INFO:tasks.rados.rados.0.smithi138.stdout:909: expect (ObjNum 369 snap 91 seq_num 369) 2023-06-27T12:42:02.676 INFO:tasks.rados.rados.0.smithi138.stdout:911: finishing write tid 1 to smithi138138488-28 2023-06-27T12:42:02.676 INFO:tasks.rados.rados.0.smithi138.stdout:911: finishing write tid 2 to smithi138138488-28 2023-06-27T12:42:02.676 INFO:tasks.rados.rados.0.smithi138.stdout:911: finishing write tid 3 to smithi138138488-28 2023-06-27T12:42:02.676 INFO:tasks.rados.rados.0.smithi138.stdout:911: finishing write tid 4 to smithi138138488-28 2023-06-27T12:42:02.677 INFO:tasks.rados.rados.0.smithi138.stdout:911: finishing write tid 5 to smithi138138488-28 2023-06-27T12:42:02.677 INFO:tasks.rados.rados.0.smithi138.stdout:911: oid 28 updating version 0 to 151 2023-06-27T12:42:02.677 INFO:tasks.rados.rados.0.smithi138.stdout:911: oid 28 version 151 is already newer than 151 2023-06-27T12:42:02.677 INFO:tasks.rados.rados.0.smithi138.stdout:911: oid 28 version 151 is already newer than 151 2023-06-27T12:42:02.677 INFO:tasks.rados.rados.0.smithi138.stdout:911: oid 28 version 151 is already newer than 150 2023-06-27T12:42:02.677 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 151 (ObjNum 442 snap 103 seq_num 442) dirty exists 2023-06-27T12:42:02.677 INFO:tasks.rados.rados.0.smithi138.stdout:911: left oid 28 (ObjNum 442 snap 103 seq_num 442) 2023-06-27T12:42:02.736 INFO:tasks.rados.rados.0.smithi138.stdout:905: done (5 left) 2023-06-27T12:42:02.736 INFO:tasks.rados.rados.0.smithi138.stdout:906: done (4 left) 2023-06-27T12:42:02.736 INFO:tasks.rados.rados.0.smithi138.stdout:909: done (3 left) 2023-06-27T12:42:02.737 INFO:tasks.rados.rados.0.smithi138.stdout:911: done (2 left) 2023-06-27T12:42:02.737 INFO:tasks.rados.rados.0.smithi138.stdout:912: done (1 left) 2023-06-27T12:42:02.737 INFO:tasks.rados.rados.0.smithi138.stdout:913: done (0 left) 2023-06-27T12:42:02.737 INFO:tasks.rados.rados.0.smithi138.stdout:914: read oid 22 snap -1 2023-06-27T12:42:02.737 INFO:tasks.rados.rados.0.smithi138.stdout:914: expect (ObjNum 428 snap 100 seq_num 428) 2023-06-27T12:42:02.737 INFO:tasks.rados.rados.0.smithi138.stdout:915: write oid 21 current snap is 103 2023-06-27T12:42:02.737 INFO:tasks.rados.rados.0.smithi138.stdout:915: seq_num 444 ranges {662308=559030,1632005=603811,2860044=602264} 2023-06-27T12:42:02.738 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:02.743 INFO:tasks.rados.rados.0.smithi138.stdout:915: writing smithi138138488-21 from 662308 to 1221338 tid 1 2023-06-27T12:42:02.749 INFO:tasks.rados.rados.0.smithi138.stdout:915: writing smithi138138488-21 from 1632005 to 2235816 tid 2 2023-06-27T12:42:02.757 INFO:tasks.rados.rados.0.smithi138.stdout:915: writing smithi138138488-21 from 2860044 to 3462308 tid 3 2023-06-27T12:42:02.758 INFO:tasks.rados.rados.0.smithi138.stdout:916: read oid 1 snap -1 2023-06-27T12:42:02.758 INFO:tasks.rados.rados.0.smithi138.stdout:916: expect (ObjNum 406 snap 98 seq_num 406) 2023-06-27T12:42:02.758 INFO:tasks.rados.rados.0.smithi138.stdout:917: delete oid 31 current snap is 103 2023-06-27T12:42:02.819 INFO:tasks.rados.rados.0.smithi138.stdout:917: done (3 left) 2023-06-27T12:42:02.819 INFO:tasks.rados.rados.0.smithi138.stdout:918: read oid 28 snap -1 2023-06-27T12:42:02.819 INFO:tasks.rados.rados.0.smithi138.stdout:918: expect (ObjNum 442 snap 103 seq_num 442) 2023-06-27T12:42:02.819 INFO:tasks.rados.rados.0.smithi138.stdout:919: delete oid 27 current snap is 103 2023-06-27T12:42:02.828 INFO:tasks.rados.rados.0.smithi138.stdout:914: expect (ObjNum 428 snap 100 seq_num 428) 2023-06-27T12:42:02.888 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:02.883+0000 7f8655acf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:02.892 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:02.888+0000 7f8655acf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:02.960 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:02.960 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:02.992 INFO:tasks.rados.rados.0.smithi138.stdout:918: expect (ObjNum 442 snap 103 seq_num 442) 2023-06-27T12:42:03.120 INFO:tasks.rados.rados.0.smithi138.stdout:915: finishing write tid 1 to smithi138138488-21 2023-06-27T12:42:03.120 INFO:tasks.rados.rados.0.smithi138.stdout:915: finishing write tid 2 to smithi138138488-21 2023-06-27T12:42:03.121 INFO:tasks.rados.rados.0.smithi138.stdout:915: finishing write tid 3 to smithi138138488-21 2023-06-27T12:42:03.121 INFO:tasks.rados.rados.0.smithi138.stdout:915: finishing write tid 4 to smithi138138488-21 2023-06-27T12:42:03.121 INFO:tasks.rados.rados.0.smithi138.stdout:915: finishing write tid 5 to smithi138138488-21 2023-06-27T12:42:03.121 INFO:tasks.rados.rados.0.smithi138.stdout:915: oid 21 updating version 0 to 261 2023-06-27T12:42:03.121 INFO:tasks.rados.rados.0.smithi138.stdout:915: oid 21 version 261 is already newer than 261 2023-06-27T12:42:03.121 INFO:tasks.rados.rados.0.smithi138.stdout:915: oid 21 version 261 is already newer than 260 2023-06-27T12:42:03.121 INFO:tasks.rados.rados.0.smithi138.stdout:915: oid 21 version 261 is already newer than 261 2023-06-27T12:42:03.122 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 21 v 261 (ObjNum 444 snap 103 seq_num 444) dirty exists 2023-06-27T12:42:03.122 INFO:tasks.rados.rados.0.smithi138.stdout:915: left oid 21 (ObjNum 444 snap 103 seq_num 444) 2023-06-27T12:42:03.122 INFO:tasks.rados.rados.0.smithi138.stdout:914: done (4 left) 2023-06-27T12:42:03.122 INFO:tasks.rados.rados.0.smithi138.stdout:915: done (3 left) 2023-06-27T12:42:03.122 INFO:tasks.rados.rados.0.smithi138.stdout:918: done (2 left) 2023-06-27T12:42:03.122 INFO:tasks.rados.rados.0.smithi138.stdout:919: done (1 left) 2023-06-27T12:42:03.122 INFO:tasks.rados.rados.0.smithi138.stdout:920: write (excl) oid 21 current snap is 103 2023-06-27T12:42:03.123 INFO:tasks.rados.rados.0.smithi138.stdout:916: expect (ObjNum 406 snap 98 seq_num 406) 2023-06-27T12:42:03.265 INFO:tasks.rados.rados.0.smithi138.stdout:920: seq_num 447 ranges {712729=784777,2091098=467491,3342313=170416} 2023-06-27T12:42:03.273 INFO:tasks.rados.rados.0.smithi138.stdout:920: writing smithi138138488-21 from 712729 to 1497506 tid 1 2023-06-27T12:42:03.280 INFO:tasks.rados.rados.0.smithi138.stdout:920: writing smithi138138488-21 from 2091098 to 2558589 tid 2 2023-06-27T12:42:03.285 INFO:tasks.rados.rados.0.smithi138.stdout:920: writing smithi138138488-21 from 3342313 to 3512729 tid 3 2023-06-27T12:42:03.286 INFO:tasks.rados.rados.0.smithi138.stdout:916: done (1 left) 2023-06-27T12:42:03.286 INFO:tasks.rados.rados.0.smithi138.stdout:921: write (excl) oid 8 current snap is 103 2023-06-27T12:42:03.286 INFO:tasks.rados.rados.0.smithi138.stdout:921: seq_num 448 ranges {729536=460026,1749580=611149,2853383=452450} 2023-06-27T12:42:03.293 INFO:tasks.rados.rados.0.smithi138.stdout:921: writing smithi138138488-8 from 729536 to 1189562 tid 1 2023-06-27T12:42:03.300 INFO:tasks.rados.rados.0.smithi138.stdout:921: writing smithi138138488-8 from 1749580 to 2360729 tid 2 2023-06-27T12:42:03.305 INFO:tasks.rados.rados.0.smithi138.stdout:921: writing smithi138138488-8 from 2853383 to 3305833 tid 3 2023-06-27T12:42:03.305 INFO:tasks.rados.rados.0.smithi138.stdout:922: snap_create 2023-06-27T12:42:03.323 INFO:tasks.rados.rados.0.smithi138.stdout:920: finishing write tid 1 to smithi138138488-21 2023-06-27T12:42:03.341 INFO:tasks.rados.rados.0.smithi138.stdout:921: finishing write tid 1 to smithi138138488-8 2023-06-27T12:42:03.373 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~88s 2023-06-27T12:42:03.375 INFO:tasks.rados.rados.0.smithi138.stdout:920: finishing write tid 2 to smithi138138488-21 2023-06-27T12:42:03.375 INFO:tasks.rados.rados.0.smithi138.stdout:920: finishing write tid 3 to smithi138138488-21 2023-06-27T12:42:03.376 INFO:tasks.rados.rados.0.smithi138.stdout:920: finishing write tid 4 to smithi138138488-21 2023-06-27T12:42:03.377 INFO:tasks.rados.rados.0.smithi138.stdout:920: finishing write tid 5 to smithi138138488-21 2023-06-27T12:42:03.377 INFO:tasks.rados.rados.0.smithi138.stdout:920: oid 21 updating version 0 to 265 2023-06-27T12:42:03.377 INFO:tasks.rados.rados.0.smithi138.stdout:920: oid 21 version 265 is already newer than 265 2023-06-27T12:42:03.377 INFO:tasks.rados.rados.0.smithi138.stdout:920: oid 21 version 265 is already newer than 265 2023-06-27T12:42:03.378 INFO:tasks.rados.rados.0.smithi138.stdout:920: oid 21 version 265 is already newer than 262 2023-06-27T12:42:03.378 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 21 v 265 (ObjNum 447 snap 103 seq_num 447) dirty exists 2023-06-27T12:42:03.378 INFO:tasks.rados.rados.0.smithi138.stdout:920: left oid 21 (ObjNum 447 snap 103 seq_num 447) 2023-06-27T12:42:03.392 INFO:tasks.rados.rados.0.smithi138.stdout:921: finishing write tid 2 to smithi138138488-8 2023-06-27T12:42:03.393 INFO:tasks.rados.rados.0.smithi138.stdout:921: finishing write tid 3 to smithi138138488-8 2023-06-27T12:42:03.393 INFO:tasks.rados.rados.0.smithi138.stdout:921: finishing write tid 4 to smithi138138488-8 2023-06-27T12:42:03.394 INFO:tasks.rados.rados.0.smithi138.stdout:921: finishing write tid 5 to smithi138138488-8 2023-06-27T12:42:03.394 INFO:tasks.rados.rados.0.smithi138.stdout:921: oid 8 updating version 0 to 54 2023-06-27T12:42:03.394 INFO:tasks.rados.rados.0.smithi138.stdout:921: oid 8 version 54 is already newer than 54 2023-06-27T12:42:03.394 INFO:tasks.rados.rados.0.smithi138.stdout:921: oid 8 version 54 is already newer than 51 2023-06-27T12:42:03.395 INFO:tasks.rados.rados.0.smithi138.stdout:921: oid 8 version 54 is already newer than 54 2023-06-27T12:42:03.395 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 8 v 54 (ObjNum 448 snap 103 seq_num 448) dirty exists 2023-06-27T12:42:03.395 INFO:tasks.rados.rados.0.smithi138.stdout:921: left oid 8 (ObjNum 448 snap 103 seq_num 448) 2023-06-27T12:42:03.546 INFO:tasks.rados.rados.0.smithi138.stdout:920: done (2 left) 2023-06-27T12:42:03.547 INFO:tasks.rados.rados.0.smithi138.stdout:921: done (1 left) 2023-06-27T12:42:03.547 INFO:tasks.rados.rados.0.smithi138.stdout:922: done (0 left) 2023-06-27T12:42:03.547 INFO:tasks.rados.rados.0.smithi138.stdout:923: read oid 48 snap -1 2023-06-27T12:42:03.547 INFO:tasks.rados.rados.0.smithi138.stdout:923: expect deleted 2023-06-27T12:42:03.547 INFO:tasks.rados.rados.0.smithi138.stdout:924: delete oid 38 current snap is 104 2023-06-27T12:42:03.599 INFO:tasks.rados.rados.0.smithi138.stdout:923: done (1 left) 2023-06-27T12:42:03.599 INFO:tasks.rados.rados.0.smithi138.stdout:924: done (0 left) 2023-06-27T12:42:03.599 INFO:tasks.rados.rados.0.smithi138.stdout:925: write oid 33 current snap is 104 2023-06-27T12:42:03.599 INFO:tasks.rados.rados.0.smithi138.stdout:925: seq_num 450 ranges {763150=726877,1982897=614818,3024582=474994} 2023-06-27T12:42:03.607 INFO:tasks.rados.rados.0.smithi138.stdout:925: writing smithi138138488-33 from 763150 to 1490027 tid 1 2023-06-27T12:42:03.613 INFO:tasks.rados.rados.0.smithi138.stdout:925: writing smithi138138488-33 from 1982897 to 2597715 tid 2 2023-06-27T12:42:03.618 INFO:tasks.rados.rados.0.smithi138.stdout:925: writing smithi138138488-33 from 3024582 to 3499576 tid 3 2023-06-27T12:42:03.619 INFO:tasks.rados.rados.0.smithi138.stdout:926: write oid 26 current snap is 104 2023-06-27T12:42:03.619 INFO:tasks.rados.rados.0.smithi138.stdout:926: seq_num 451 ranges {779957=402126,1925026=758476,3335652=244305} 2023-06-27T12:42:03.625 INFO:tasks.rados.rados.0.smithi138.stdout:926: writing smithi138138488-26 from 779957 to 1182083 tid 1 2023-06-27T12:42:03.633 INFO:tasks.rados.rados.0.smithi138.stdout:926: writing smithi138138488-26 from 1925026 to 2683502 tid 2 2023-06-27T12:42:03.638 INFO:tasks.rados.rados.0.smithi138.stdout:926: writing smithi138138488-26 from 3335652 to 3579957 tid 3 2023-06-27T12:42:03.639 INFO:tasks.rados.rados.0.smithi138.stdout:927: write (excl) oid 27 current snap is 104 2023-06-27T12:42:03.639 INFO:tasks.rados.rados.0.smithi138.stdout:925: finishing write tid 1 to smithi138138488-33 2023-06-27T12:42:03.639 INFO:tasks.rados.rados.0.smithi138.stdout:927: seq_num 452 ranges {796764=477375,1983508=618487,3363075=233689} 2023-06-27T12:42:03.648 INFO:tasks.rados.rados.0.smithi138.stdout:927: writing smithi138138488-27 from 796764 to 1274139 tid 1 2023-06-27T12:42:03.656 INFO:tasks.rados.rados.0.smithi138.stdout:927: writing smithi138138488-27 from 1983508 to 2601995 tid 2 2023-06-27T12:42:03.662 INFO:tasks.rados.rados.0.smithi138.stdout:927: writing smithi138138488-27 from 3363075 to 3596764 tid 3 2023-06-27T12:42:03.663 INFO:tasks.rados.rados.0.smithi138.stdout:928: write (excl) oid 19 current snap is 104 2023-06-27T12:42:03.663 INFO:tasks.rados.rados.0.smithi138.stdout:926: finishing write tid 1 to smithi138138488-26 2023-06-27T12:42:03.663 INFO:tasks.rados.rados.0.smithi138.stdout:928: seq_num 453 ranges {413571=552624,1641990=762145,2990498=425163} 2023-06-27T12:42:03.668 INFO:tasks.rados.rados.0.smithi138.stdout:928: writing smithi138138488-19 from 413571 to 966195 tid 1 2023-06-27T12:42:03.676 INFO:tasks.rados.rados.0.smithi138.stdout:928: writing smithi138138488-19 from 1641990 to 2404135 tid 2 2023-06-27T12:42:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:928: writing smithi138138488-19 from 2990498 to 3415661 tid 3 2023-06-27T12:42:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:929: delete oid 21 current snap is 104 2023-06-27T12:42:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:925: finishing write tid 2 to smithi138138488-33 2023-06-27T12:42:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:925: finishing write tid 3 to smithi138138488-33 2023-06-27T12:42:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:925: finishing write tid 4 to smithi138138488-33 2023-06-27T12:42:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:925: finishing write tid 5 to smithi138138488-33 2023-06-27T12:42:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:925: oid 33 updating version 0 to 100 2023-06-27T12:42:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:925: oid 33 version 100 is already newer than 100 2023-06-27T12:42:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:925: oid 33 version 100 is already newer than 99 2023-06-27T12:42:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:925: oid 33 version 100 is already newer than 100 2023-06-27T12:42:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 100 (ObjNum 450 snap 104 seq_num 450) dirty exists 2023-06-27T12:42:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:925: left oid 33 (ObjNum 450 snap 104 seq_num 450) 2023-06-27T12:42:03.699 INFO:tasks.rados.rados.0.smithi138.stdout:928: finishing write tid 1 to smithi138138488-19 2023-06-27T12:42:03.702 INFO:tasks.rados.rados.0.smithi138.stdout:926: finishing write tid 2 to smithi138138488-26 2023-06-27T12:42:03.744 INFO:tasks.rados.rados.0.smithi138.stdout:926: finishing write tid 3 to smithi138138488-26 2023-06-27T12:42:03.744 INFO:tasks.rados.rados.0.smithi138.stdout:926: finishing write tid 4 to smithi138138488-26 2023-06-27T12:42:03.744 INFO:tasks.rados.rados.0.smithi138.stdout:927: finishing write tid 1 to smithi138138488-27 2023-06-27T12:42:03.744 INFO:tasks.rados.rados.0.smithi138.stdout:926: finishing write tid 5 to smithi138138488-26 2023-06-27T12:42:03.745 INFO:tasks.rados.rados.0.smithi138.stdout:926: oid 26 updating version 0 to 133 2023-06-27T12:42:03.745 INFO:tasks.rados.rados.0.smithi138.stdout:926: oid 26 version 133 is already newer than 133 2023-06-27T12:42:03.745 INFO:tasks.rados.rados.0.smithi138.stdout:926: oid 26 version 133 is already newer than 133 2023-06-27T12:42:03.745 INFO:tasks.rados.rados.0.smithi138.stdout:926: oid 26 version 133 is already newer than 133 2023-06-27T12:42:03.745 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 133 (ObjNum 451 snap 104 seq_num 451) dirty exists 2023-06-27T12:42:03.745 INFO:tasks.rados.rados.0.smithi138.stdout:926: left oid 26 (ObjNum 451 snap 104 seq_num 451) 2023-06-27T12:42:03.755 INFO:tasks.rados.rados.0.smithi138.stdout:927: finishing write tid 2 to smithi138138488-27 2023-06-27T12:42:03.756 INFO:tasks.rados.rados.0.smithi138.stdout:928: finishing write tid 2 to smithi138138488-19 2023-06-27T12:42:03.756 INFO:tasks.rados.rados.0.smithi138.stdout:927: finishing write tid 3 to smithi138138488-27 2023-06-27T12:42:03.756 INFO:tasks.rados.rados.0.smithi138.stdout:928: finishing write tid 3 to smithi138138488-19 2023-06-27T12:42:03.757 INFO:tasks.rados.rados.0.smithi138.stdout:928: finishing write tid 4 to smithi138138488-19 2023-06-27T12:42:03.757 INFO:tasks.rados.rados.0.smithi138.stdout:927: finishing write tid 4 to smithi138138488-27 2023-06-27T12:42:03.758 INFO:tasks.rados.rados.0.smithi138.stdout:928: finishing write tid 5 to smithi138138488-19 2023-06-27T12:42:03.758 INFO:tasks.rados.rados.0.smithi138.stdout:928: oid 19 updating version 0 to 266 2023-06-27T12:42:03.759 INFO:tasks.rados.rados.0.smithi138.stdout:928: oid 19 updating version 266 to 270 2023-06-27T12:42:03.759 INFO:tasks.rados.rados.0.smithi138.stdout:928: oid 19 version 270 is already newer than 270 2023-06-27T12:42:03.759 INFO:tasks.rados.rados.0.smithi138.stdout:928: oid 19 version 270 is already newer than 270 2023-06-27T12:42:03.759 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 270 (ObjNum 453 snap 104 seq_num 453) dirty exists 2023-06-27T12:42:03.759 INFO:tasks.rados.rados.0.smithi138.stdout:928: left oid 19 (ObjNum 453 snap 104 seq_num 453) 2023-06-27T12:42:03.759 INFO:tasks.rados.rados.0.smithi138.stdout:927: finishing write tid 5 to smithi138138488-27 2023-06-27T12:42:03.759 INFO:tasks.rados.rados.0.smithi138.stdout:927: oid 27 updating version 0 to 141 2023-06-27T12:42:03.759 INFO:tasks.rados.rados.0.smithi138.stdout:927: oid 27 version 141 is already newer than 141 2023-06-27T12:42:03.760 INFO:tasks.rados.rados.0.smithi138.stdout:927: oid 27 version 141 is already newer than 138 2023-06-27T12:42:03.760 INFO:tasks.rados.rados.0.smithi138.stdout:927: oid 27 version 141 is already newer than 141 2023-06-27T12:42:03.760 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 141 (ObjNum 452 snap 104 seq_num 452) dirty exists 2023-06-27T12:42:03.760 INFO:tasks.rados.rados.0.smithi138.stdout:927: left oid 27 (ObjNum 452 snap 104 seq_num 452) 2023-06-27T12:42:03.837 INFO:tasks.rados.rados.0.smithi138.stdout:925: done (4 left) 2023-06-27T12:42:03.837 INFO:tasks.rados.rados.0.smithi138.stdout:926: done (3 left) 2023-06-27T12:42:03.838 INFO:tasks.rados.rados.0.smithi138.stdout:927: done (2 left) 2023-06-27T12:42:03.838 INFO:tasks.rados.rados.0.smithi138.stdout:928: done (1 left) 2023-06-27T12:42:03.838 INFO:tasks.rados.rados.0.smithi138.stdout:929: done (0 left) 2023-06-27T12:42:03.838 INFO:tasks.rados.rados.0.smithi138.stdout:930: delete oid 43 current snap is 104 2023-06-27T12:42:03.885 INFO:tasks.rados.rados.0.smithi138.stdout:930: done (0 left) 2023-06-27T12:42:03.885 INFO:tasks.rados.rados.0.smithi138.stdout:931: snap_create 2023-06-27T12:42:04.555 INFO:tasks.rados.rados.0.smithi138.stdout:931: done (0 left) 2023-06-27T12:42:04.556 INFO:tasks.rados.rados.0.smithi138.stdout:932: read oid 42 snap -1 2023-06-27T12:42:04.556 INFO:tasks.rados.rados.0.smithi138.stdout:932: expect (ObjNum 427 snap 100 seq_num 427) 2023-06-27T12:42:04.556 INFO:tasks.rados.rados.0.smithi138.stdout:933: read oid 13 snap 103 2023-06-27T12:42:04.556 INFO:tasks.rados.rados.0.smithi138.stdout:933: expect deleted 2023-06-27T12:42:04.556 INFO:tasks.rados.rados.0.smithi138.stdout:934: read oid 41 snap 104 2023-06-27T12:42:04.556 INFO:tasks.rados.rados.0.smithi138.stdout:934: expect (ObjNum 268 snap 70 seq_num 268) 2023-06-27T12:42:04.557 INFO:tasks.rados.rados.0.smithi138.stdout:935: write (excl) oid 48 current snap is 105 2023-06-27T12:42:04.557 INFO:tasks.rados.rados.0.smithi138.stdout:935: seq_num 456 ranges {463992=778371,1701083=625825,2956414=658979} 2023-06-27T12:42:04.562 INFO:tasks.rados.rados.0.smithi138.stdout:935: writing smithi138138488-48 from 463992 to 1242363 tid 1 2023-06-27T12:42:04.568 INFO:tasks.rados.rados.0.smithi138.stdout:935: writing smithi138138488-48 from 1701083 to 2326908 tid 2 2023-06-27T12:42:04.575 INFO:tasks.rados.rados.0.smithi138.stdout:935: writing smithi138138488-48 from 2956414 to 3615393 tid 3 2023-06-27T12:42:04.575 INFO:tasks.rados.rados.0.smithi138.stdout:936: write (excl) oid 8 current snap is 105 2023-06-27T12:42:04.576 INFO:tasks.rados.rados.0.smithi138.stdout:936: seq_num 457 ranges {480799=569973,1475918=769483,2700190=586604} 2023-06-27T12:42:04.581 INFO:tasks.rados.rados.0.smithi138.stdout:936: writing smithi138138488-8 from 480799 to 1050772 tid 1 2023-06-27T12:42:04.588 INFO:tasks.rados.rados.0.smithi138.stdout:936: writing smithi138138488-8 from 1475918 to 2245401 tid 2 2023-06-27T12:42:04.594 INFO:tasks.rados.rados.0.smithi138.stdout:936: writing smithi138138488-8 from 2700190 to 3286794 tid 3 2023-06-27T12:42:04.594 INFO:tasks.rados.rados.0.smithi138.stdout:937: snap_create 2023-06-27T12:42:04.614 INFO:tasks.rados.rados.0.smithi138.stdout:935: finishing write tid 1 to smithi138138488-48 2023-06-27T12:42:04.631 INFO:tasks.rados.rados.0.smithi138.stdout:935: finishing write tid 2 to smithi138138488-48 2023-06-27T12:42:04.655 INFO:tasks.rados.rados.0.smithi138.stdout:934: expect (ObjNum 268 snap 70 seq_num 268) 2023-06-27T12:42:04.826 INFO:tasks.rados.rados.0.smithi138.stdout:932: expect (ObjNum 427 snap 100 seq_num 427) 2023-06-27T12:42:04.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:04.989 INFO:tasks.rados.rados.0.smithi138.stdout:936: finishing write tid 1 to smithi138138488-8 2023-06-27T12:42:04.990 INFO:tasks.rados.rados.0.smithi138.stdout:935: finishing write tid 3 to smithi138138488-48 2023-06-27T12:42:04.990 INFO:tasks.rados.rados.0.smithi138.stdout:935: finishing write tid 4 to smithi138138488-48 2023-06-27T12:42:04.990 INFO:tasks.rados.rados.0.smithi138.stdout:935: finishing write tid 5 to smithi138138488-48 2023-06-27T12:42:04.990 INFO:tasks.rados.rados.0.smithi138.stdout:935: oid 48 updating version 0 to 163 2023-06-27T12:42:04.990 INFO:tasks.rados.rados.0.smithi138.stdout:935: oid 48 version 163 is already newer than 163 2023-06-27T12:42:04.990 INFO:tasks.rados.rados.0.smithi138.stdout:935: oid 48 version 163 is already newer than 160 2023-06-27T12:42:04.990 INFO:tasks.rados.rados.0.smithi138.stdout:935: oid 48 version 163 is already newer than 163 2023-06-27T12:42:04.990 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 163 (ObjNum 456 snap 105 seq_num 456) dirty exists 2023-06-27T12:42:04.991 INFO:tasks.rados.rados.0.smithi138.stdout:935: left oid 48 (ObjNum 456 snap 105 seq_num 456) 2023-06-27T12:42:04.991 INFO:tasks.rados.rados.0.smithi138.stdout:936: finishing write tid 2 to smithi138138488-8 2023-06-27T12:42:04.991 INFO:tasks.rados.rados.0.smithi138.stdout:936: finishing write tid 3 to smithi138138488-8 2023-06-27T12:42:04.991 INFO:tasks.rados.rados.0.smithi138.stdout:936: finishing write tid 4 to smithi138138488-8 2023-06-27T12:42:04.991 INFO:tasks.rados.rados.0.smithi138.stdout:936: finishing write tid 5 to smithi138138488-8 2023-06-27T12:42:04.991 INFO:tasks.rados.rados.0.smithi138.stdout:936: oid 8 updating version 0 to 55 2023-06-27T12:42:04.991 INFO:tasks.rados.rados.0.smithi138.stdout:936: oid 8 updating version 55 to 59 2023-06-27T12:42:04.992 INFO:tasks.rados.rados.0.smithi138.stdout:936: oid 8 version 59 is already newer than 59 2023-06-27T12:42:04.992 INFO:tasks.rados.rados.0.smithi138.stdout:936: oid 8 version 59 is already newer than 59 2023-06-27T12:42:04.992 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 8 v 59 (ObjNum 457 snap 105 seq_num 457) dirty exists 2023-06-27T12:42:04.992 INFO:tasks.rados.rados.0.smithi138.stdout:936: left oid 8 (ObjNum 457 snap 105 seq_num 457) 2023-06-27T12:42:05.089 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:05.084+0000 7f2c77c0a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:05.091 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:05.088+0000 7f2c77c0a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:05.135 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:05.138 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:05.139 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:05.426 INFO:tasks.rados.rados.0.smithi138.stdout:932: done (5 left) 2023-06-27T12:42:05.426 INFO:tasks.rados.rados.0.smithi138.stdout:933: done (4 left) 2023-06-27T12:42:05.426 INFO:tasks.rados.rados.0.smithi138.stdout:934: done (3 left) 2023-06-27T12:42:05.426 INFO:tasks.rados.rados.0.smithi138.stdout:935: done (2 left) 2023-06-27T12:42:05.426 INFO:tasks.rados.rados.0.smithi138.stdout:936: done (1 left) 2023-06-27T12:42:05.427 INFO:tasks.rados.rados.0.smithi138.stdout:937: done (0 left) 2023-06-27T12:42:05.427 INFO:tasks.rados.rados.0.smithi138.stdout:938: snap_create 2023-06-27T12:42:05.655 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:42:05.655 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:42:05.656 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:05.785 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:05.778+0000 7f3caad12700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:05.788 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:05.784+0000 7f3caad12700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:06.002 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:06.005 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:06.016 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2023-06-27T12:42:05.957408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80574342,"num_objects":225,"num_object_clones":182,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1360,"num_read_kb":937766,"num_write":1171,"num_write_kb":339294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2307,"ondisk_log_size":2307,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":761456,"kb_used_data":633072,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376725904,"statfs":{"total":386547056640,"available":385767325696,"internally_reserved":0,"allocated":648265728,"data_stored":734949317,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5092653,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":44,"num_read_kb":24189,"num_write":50,"num_write_kb":15300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000965"},"pg_stats":[{"pgid":"3.f","version":"223'14","reported_seq":541,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.571374+0000","last_change":"2023-06-27T12:42:02.562076+0000","last_active":"2023-06-27T12:42:03.571374+0000","last_peered":"2023-06-27T12:42:03.571374+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:03.571374+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":6,"num_read_kb":3,"num_write":13,"num_write_kb":3512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"230'141","reported_seq":721,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.708757+0000","last_change":"2023-06-27T12:42:02.552888+0000","last_active":"2023-06-27T12:42:03.708757+0000","last_peered":"2023-06-27T12:42:03.708757+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:03.708757+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748258,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":100,"num_read_kb":107984,"num_write":96,"num_write_kb":27863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"219'169","reported_seq":711,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.560324+0000","last_change":"2023-06-27T12:42:02.870028+0000","last_active":"2023-06-27T12:42:03.560324+0000","last_peered":"2023-06-27T12:42:03.560324+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:03.560324+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933069,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":119,"num_read_kb":55218,"num_write":116,"num_write_kb":34162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"230'100","reported_seq":675,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.632026+0000","last_change":"2023-06-27T12:42:02.585694+0000","last_active":"2023-06-27T12:42:03.632026+0000","last_peered":"2023-06-27T12:42:03.632026+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:03.632026+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9865907,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":82,"num_read_kb":40231,"num_write":79,"num_write_kb":22429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"229'54","reported_seq":242,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.573149+0000","last_change":"2023-06-27T12:42:02.564056+0000","last_active":"2023-06-27T12:42:03.573149+0000","last_peered":"2023-06-27T12:42:03.573149+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:03.573149+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571570,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":24676,"num_write":24,"num_write_kb":7037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"229'284","reported_seq":727,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-06-27T12:42:03.574349+0000","last_change":"2023-06-27T12:42:02.564640+0000","last_active":"2023-06-27T12:42:03.574349+0000","last_peered":"2023-06-27T12:42:03.574349+0000","last_clean":"2023-06-27T12:42:03.574349+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:03.574349+0000","last_undegraded":"2023-06-27T12:42:03.574349+0000","last_fullsized":"2023-06-27T12:42:03.574349+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2453888,"num_objects":32,"num_object_clones":26,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":235,"num_read_kb":149308,"num_write":161,"num_write_kb":45897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"229'151","reported_seq":596,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-06-27T12:42:03.576061+0000","last_change":"2023-06-27T12:42:02.585428+0000","last_active":"2023-06-27T12:42:03.576061+0000","last_peered":"2023-06-27T12:42:03.576061+0000","last_clean":"2023-06-27T12:42:03.576061+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:03.576061+0000","last_undegraded":"2023-06-27T12:42:03.576061+0000","last_fullsized":"2023-06-27T12:42:03.576061+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5630477,"num_objects":18,"num_object_clones":15,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":109,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"199'60","reported_seq":602,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.557547+0000","last_change":"2023-06-27T12:42:02.784652+0000","last_active":"2023-06-27T12:42:03.557547+0000","last_peered":"2023-06-27T12:42:03.557547+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:03.557547+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916064,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":374,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-06-27T12:42:03.556833+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:03.556833+0000","last_peered":"2023-06-27T12:42:03.556833+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:03.556833+0000","last_undegraded":"2023-06-27T12:42:03.556833+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":524,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-06-27T12:42:03.558033+0000","last_change":"2023-06-27T12:42:02.866418+0000","last_active":"2023-06-27T12:42:03.558033+0000","last_peered":"2023-06-27T12:42:03.558033+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:03.558033+0000","last_undegraded":"2023-06-27T12:42:03.558033+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":355,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-06-27T12:42:03.572162+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:03.572162+0000","last_peered":"2023-06-27T12:42:03.572162+0000","last_clean":"2023-06-27T12:42:03.572162+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:03.572162+0000","last_undegraded":"2023-06-27T12:42:03.572162+0000","last_fullsized":"2023-06-27T12:42:03.572162+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"226'127","reported_seq":674,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.555499+0000","last_change":"2023-06-27T12:42:02.553396+0000","last_active":"2023-06-27T12:42:03.555499+0000","last_peered":"2023-06-27T12:42:03.555499+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:03.555499+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6722932,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":56879,"num_write":83,"num_write_kb":22643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":356,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-06-27T12:42:03.555678+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:03.555678+0000","last_peered":"2023-06-27T12:42:03.555678+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:03.555678+0000","last_undegraded":"2023-06-27T12:42:03.555678+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"225'316","reported_seq":301,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.564159+0000","last_change":"2023-06-27T12:42:03.564159+0000","last_active":"2023-06-27T12:42:03.564159+0000","last_peered":"2023-06-27T12:42:03.564159+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:03.564159+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":28,"num_read_kb":17194,"num_write":46,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":353,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-06-27T12:42:03.558668+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:03.558668+0000","last_peered":"2023-06-27T12:42:03.558668+0000","last_clean":"2023-06-27T12:42:03.558668+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:03.558668+0000","last_undegraded":"2023-06-27T12:42:03.558668+0000","last_fullsized":"2023-06-27T12:42:03.558668+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":107,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-06-27T12:42:03.573328+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:03.573328+0000","last_peered":"2023-06-27T12:42:03.573328+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:03.573328+0000","last_undegraded":"2023-06-27T12:42:03.573328+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"230'270","reported_seq":849,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.754917+0000","last_change":"2023-06-27T12:42:02.778354+0000","last_active":"2023-06-27T12:42:03.754917+0000","last_peered":"2023-06-27T12:42:03.754917+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:03.754917+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12487799,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":245,"num_read_kb":150311,"num_write":202,"num_write_kb":59110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":355,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-06-27T12:42:03.571790+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:03.571790+0000","last_peered":"2023-06-27T12:42:03.571790+0000","last_clean":"2023-06-27T12:42:03.571790+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:03.571790+0000","last_undegraded":"2023-06-27T12:42:03.571790+0000","last_fullsized":"2023-06-27T12:42:03.571790+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"220'143","reported_seq":264,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.564164+0000","last_change":"2023-06-27T12:42:02.865316+0000","last_active":"2023-06-27T12:42:03.564164+0000","last_peered":"2023-06-27T12:42:03.564164+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:03.564164+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117781,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":60,"num_read_kb":35684,"num_write":33,"num_write_kb":9435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-06-27T12:42:03.554506+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:03.554506+0000","last_peered":"2023-06-27T12:42:03.554506+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:03.554506+0000","last_undegraded":"2023-06-27T12:42:03.554506+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"230'133","reported_seq":700,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.701705+0000","last_change":"2023-06-27T12:42:02.554978+0000","last_active":"2023-06-27T12:42:03.701705+0000","last_peered":"2023-06-27T12:42:03.701705+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:03.701705+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9193594,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":80,"num_read_kb":84631,"num_write":96,"num_write_kb":28435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"218'93","reported_seq":259,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.562599+0000","last_change":"2023-06-27T12:42:02.781089+0000","last_active":"2023-06-27T12:42:03.562599+0000","last_peered":"2023-06-27T12:42:03.562599+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:03.562599+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2739607,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":50,"num_read_kb":26195,"num_write":26,"num_write_kb":7438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":374,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-06-27T12:42:03.558907+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:03.558907+0000","last_peered":"2023-06-27T12:42:03.558907+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:03.558907+0000","last_undegraded":"2023-06-27T12:42:03.558907+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":353,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-06-27T12:42:03.565319+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:03.565319+0000","last_peered":"2023-06-27T12:42:03.565319+0000","last_clean":"2023-06-27T12:42:03.565319+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:03.565319+0000","last_undegraded":"2023-06-27T12:42:03.565319+0000","last_fullsized":"2023-06-27T12:42:03.565319+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"226'155","reported_seq":683,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:03.563871+0000","last_change":"2023-06-27T12:42:03.563871+0000","last_active":"2023-06-27T12:42:03.563871+0000","last_peered":"2023-06-27T12:42:03.563871+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:03.563871+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3193396,"num_objects":17,"num_object_clones":14,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":74,"num_read_kb":68218,"num_write":74,"num_write_kb":21207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":80574342,"num_objects":225,"num_object_clones":182,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1360,"num_read_kb":937766,"num_write":1171,"num_write_kb":339294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2210,"ondisk_log_size":2210,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673210,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":180888,"kb_used_data":148868,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190952,"statfs":{"total":96636764160,"available":96451534848,"internally_reserved":0,"allocated":152440832,"data_stored":173043476,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":196576,"kb_used_data":164492,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94175264,"statfs":{"total":96636764160,"available":96435470336,"internally_reserved":0,"allocated":168439808,"data_stored":189844210,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673210,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":202712,"kb_used_data":169352,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94169128,"statfs":{"total":96636764160,"available":96429187072,"internally_reserved":0,"allocated":173416448,"data_stored":195022079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:06.018 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.d 3.c 3.b 3.8 3.3 3.2 3.4 3.6 2023-06-27T12:42:06.154 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:06.138+0000 7f3f4dec9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:06.157 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:06.153+0000 7f3f4dec9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:06.351 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.c] on osd.0 to force-recovery; instructing pg(s) [3.2,3.8,3.d] on osd.1 to force-recovery; instructing pg(s) [3.3,3.4,3.6] on osd.2 to force-recovery; 2023-06-27T12:42:06.351 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:42:06.431 INFO:tasks.rados.rados.0.smithi138.stdout:938: done (0 left) 2023-06-27T12:42:06.432 INFO:tasks.rados.rados.0.smithi138.stdout:939: write (excl) oid 44 current snap is 107 2023-06-27T12:42:06.432 INFO:tasks.rados.rados.0.smithi138.stdout:939: seq_num 458 ranges {497606=645222,1934400=629494,3243966=453640} 2023-06-27T12:42:06.438 INFO:tasks.rados.rados.0.smithi138.stdout:939: writing smithi138138488-44 from 497606 to 1142828 tid 1 2023-06-27T12:42:06.446 INFO:tasks.rados.rados.0.smithi138.stdout:939: writing smithi138138488-44 from 1934400 to 2563894 tid 2 2023-06-27T12:42:06.452 INFO:tasks.rados.rados.0.smithi138.stdout:939: writing smithi138138488-44 from 3243966 to 3697606 tid 3 2023-06-27T12:42:06.453 INFO:tasks.rados.rados.0.smithi138.stdout:940: read oid 11 snap 105 2023-06-27T12:42:06.453 INFO:tasks.rados.rados.0.smithi138.stdout:940: expect (ObjNum 389 snap 95 seq_num 389) 2023-06-27T12:42:06.453 INFO:tasks.rados.rados.0.smithi138.stdout:941: read oid 48 snap -1 2023-06-27T12:42:06.453 INFO:tasks.rados.rados.0.smithi138.stdout:941: expect (ObjNum 456 snap 105 seq_num 456) 2023-06-27T12:42:06.453 INFO:tasks.rados.rados.0.smithi138.stdout:942: snap_remove snap 103 2023-06-27T12:42:07.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:07.267 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:07.262+0000 7f3617dc8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:07.270 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:07.266+0000 7f3617dc8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:07.315 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:07.318 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:07.319 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:07.433 INFO:tasks.rados.rados.0.smithi138.stdout:942: done (3 left) 2023-06-27T12:42:07.434 INFO:tasks.rados.rados.0.smithi138.stdout:943: snap_remove snap 106 2023-06-27T12:42:08.440 INFO:tasks.rados.rados.0.smithi138.stdout:943: done (3 left) 2023-06-27T12:42:08.441 INFO:tasks.rados.rados.0.smithi138.stdout:944: write oid 19 current snap is 107 2023-06-27T12:42:08.441 INFO:tasks.rados.rados.0.smithi138.stdout:944: seq_num 459 ranges {514413=720471,1876529=773152,3438683=275730} 2023-06-27T12:42:08.447 INFO:tasks.rados.rados.0.smithi138.stdout:944: writing smithi138138488-19 from 514413 to 1234884 tid 1 2023-06-27T12:42:08.455 INFO:tasks.rados.rados.0.smithi138.stdout:944: writing smithi138138488-19 from 1876529 to 2649681 tid 2 2023-06-27T12:42:08.462 INFO:tasks.rados.rados.0.smithi138.stdout:944: writing smithi138138488-19 from 3438683 to 3714413 tid 3 2023-06-27T12:42:08.462 INFO:tasks.rados.rados.0.smithi138.stdout:945: snap_remove snap 104 2023-06-27T12:42:08.877 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~94s 2023-06-27T12:42:09.320 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:945: done (4 left) 2023-06-27T12:42:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:946: snap_create 2023-06-27T12:42:09.450 INFO:tasks.rados.rados.0.smithi138.stdout:941: expect (ObjNum 456 snap 105 seq_num 456) 2023-06-27T12:42:09.461 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:09.456+0000 7f498948c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:09.461+0000 7f498948c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:09.516 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:09.519 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:09.519 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:09.596 INFO:tasks.rados.rados.0.smithi138.stdout:940: expect (ObjNum 389 snap 95 seq_num 389) 2023-06-27T12:42:09.691 INFO:tasks.rados.rados.0.smithi138.stdout:939: finishing write tid 1 to smithi138138488-44 2023-06-27T12:42:09.691 INFO:tasks.rados.rados.0.smithi138.stdout:939: finishing write tid 2 to smithi138138488-44 2023-06-27T12:42:09.691 INFO:tasks.rados.rados.0.smithi138.stdout:939: finishing write tid 3 to smithi138138488-44 2023-06-27T12:42:09.692 INFO:tasks.rados.rados.0.smithi138.stdout:939: finishing write tid 4 to smithi138138488-44 2023-06-27T12:42:09.692 INFO:tasks.rados.rados.0.smithi138.stdout:939: finishing write tid 5 to smithi138138488-44 2023-06-27T12:42:09.692 INFO:tasks.rados.rados.0.smithi138.stdout:939: oid 44 updating version 0 to 289 2023-06-27T12:42:09.692 INFO:tasks.rados.rados.0.smithi138.stdout:939: oid 44 version 289 is already newer than 289 2023-06-27T12:42:09.692 INFO:tasks.rados.rados.0.smithi138.stdout:939: oid 44 version 289 is already newer than 285 2023-06-27T12:42:09.692 INFO:tasks.rados.rados.0.smithi138.stdout:939: oid 44 version 289 is already newer than 289 2023-06-27T12:42:09.693 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 289 (ObjNum 458 snap 107 seq_num 458) dirty exists 2023-06-27T12:42:09.693 INFO:tasks.rados.rados.0.smithi138.stdout:939: left oid 44 (ObjNum 458 snap 107 seq_num 458) 2023-06-27T12:42:09.693 INFO:tasks.rados.rados.0.smithi138.stdout:944: finishing write tid 1 to smithi138138488-19 2023-06-27T12:42:09.693 INFO:tasks.rados.rados.0.smithi138.stdout:944: finishing write tid 2 to smithi138138488-19 2023-06-27T12:42:09.693 INFO:tasks.rados.rados.0.smithi138.stdout:944: finishing write tid 3 to smithi138138488-19 2023-06-27T12:42:09.693 INFO:tasks.rados.rados.0.smithi138.stdout:944: finishing write tid 4 to smithi138138488-19 2023-06-27T12:42:09.693 INFO:tasks.rados.rados.0.smithi138.stdout:944: finishing write tid 5 to smithi138138488-19 2023-06-27T12:42:09.694 INFO:tasks.rados.rados.0.smithi138.stdout:944: oid 19 updating version 0 to 281 2023-06-27T12:42:09.694 INFO:tasks.rados.rados.0.smithi138.stdout:944: oid 19 version 281 is already newer than 281 2023-06-27T12:42:09.694 INFO:tasks.rados.rados.0.smithi138.stdout:944: oid 19 version 281 is already newer than 281 2023-06-27T12:42:09.694 INFO:tasks.rados.rados.0.smithi138.stdout:944: oid 19 version 281 is already newer than 281 2023-06-27T12:42:09.694 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 281 (ObjNum 459 snap 107 seq_num 459) dirty exists 2023-06-27T12:42:09.694 INFO:tasks.rados.rados.0.smithi138.stdout:944: left oid 19 (ObjNum 459 snap 107 seq_num 459) 2023-06-27T12:42:10.458 INFO:tasks.rados.rados.0.smithi138.stdout:939: done (4 left) 2023-06-27T12:42:10.458 INFO:tasks.rados.rados.0.smithi138.stdout:940: done (3 left) 2023-06-27T12:42:10.458 INFO:tasks.rados.rados.0.smithi138.stdout:941: done (2 left) 2023-06-27T12:42:10.459 INFO:tasks.rados.rados.0.smithi138.stdout:944: done (1 left) 2023-06-27T12:42:10.459 INFO:tasks.rados.rados.0.smithi138.stdout:946: done (0 left) 2023-06-27T12:42:10.459 INFO:tasks.rados.rados.0.smithi138.stdout:947: snap_remove snap 105 2023-06-27T12:42:11.363 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:42:11.363 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:42:11.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:11.459 INFO:tasks.rados.rados.0.smithi138.stdout:947: done (0 left) 2023-06-27T12:42:11.459 INFO:tasks.rados.rados.0.smithi138.stdout:948: delete oid 19 current snap is 108 2023-06-27T12:42:11.507 INFO:tasks.rados.rados.0.smithi138.stdout:948: done (0 left) 2023-06-27T12:42:11.507 INFO:tasks.rados.rados.0.smithi138.stdout:949: write oid 48 current snap is 108 2023-06-27T12:42:11.508 INFO:tasks.rados.rados.0.smithi138.stdout:949: seq_num 461 ranges {548027=470969,1593493=776821,3093529=631692} 2023-06-27T12:42:11.508 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:11.500+0000 7fb6e5604700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:11.511 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:11.507+0000 7fb6e5604700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:11.513 INFO:tasks.rados.rados.0.smithi138.stdout:949: writing smithi138138488-48 from 548027 to 1018996 tid 1 2023-06-27T12:42:11.520 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:11.521 INFO:tasks.rados.rados.0.smithi138.stdout:949: writing smithi138138488-48 from 1593493 to 2370314 tid 2 2023-06-27T12:42:11.528 INFO:tasks.rados.rados.0.smithi138.stdout:949: writing smithi138138488-48 from 3093529 to 3725221 tid 3 2023-06-27T12:42:11.529 INFO:tasks.rados.rados.0.smithi138.stdout:950: delete oid 50 current snap is 108 2023-06-27T12:42:11.569 INFO:tasks.rados.rados.0.smithi138.stdout:950: done (1 left) 2023-06-27T12:42:11.569 INFO:tasks.rados.rados.0.smithi138.stdout:951: snap_remove snap 107 2023-06-27T12:42:11.651 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:11.646+0000 7f4af8037700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:11.654 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:11.650+0000 7f4af8037700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:11.701 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:11.705 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:11.706 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:11.740 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:11.742 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:11.754 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2023-06-27T12:42:11.453544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":85734157,"num_objects":228,"num_object_clones":185,"num_object_copies":684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":1385,"num_read_kb":962059,"num_write":1191,"num_write_kb":346613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2340,"ondisk_log_size":2340,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":753252,"kb_used_data":629572,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376734108,"statfs":{"total":386547056640,"available":385775726592,"internally_reserved":0,"allocated":644681728,"data_stored":731462190,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5159815,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":25,"num_read_kb":24293,"num_write":20,"num_write_kb":7319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023858"},"pg_stats":[{"pgid":"3.f","version":"230'16","reported_seq":550,"reported_epoch":234,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:07.442318+0000","last_change":"2023-06-27T12:42:07.465355+0000","last_active":"2023-06-27T12:42:07.442318+0000","last_peered":"2023-06-27T12:42:07.442318+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:07.442318+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":6,"num_read_kb":3,"num_write":13,"num_write_kb":3512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"230'141","reported_seq":731,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:08.466186+0000","last_change":"2023-06-27T12:42:08.474972+0000","last_active":"2023-06-27T12:42:08.466186+0000","last_peered":"2023-06-27T12:42:08.466186+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:08.466186+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748258,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":100,"num_read_kb":107984,"num_write":96,"num_write_kb":27863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"219'169","reported_seq":724,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:08.522876+0000","last_change":"2023-06-27T12:42:08.542874+0000","last_active":"2023-06-27T12:42:08.522876+0000","last_peered":"2023-06-27T12:42:08.522876+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:08.522876+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933069,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":61153,"num_write":116,"num_write_kb":34162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"230'100","reported_seq":687,"reported_epoch":234,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:07.436160+0000","last_change":"2023-06-27T12:42:07.474087+0000","last_active":"2023-06-27T12:42:07.436160+0000","last_peered":"2023-06-27T12:42:07.436160+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:07.436160+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9865907,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":89,"num_read_kb":48099,"num_write":79,"num_write_kb":22429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"231'59","reported_seq":256,"reported_epoch":234,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:07.441815+0000","last_change":"2023-06-27T12:42:07.470447+0000","last_active":"2023-06-27T12:42:07.441815+0000","last_peered":"2023-06-27T12:42:07.441815+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:07.441815+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4722833,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":24677,"num_write":29,"num_write_kb":8919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"233'289","reported_seq":741,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-06-27T12:42:07.439606+0000","last_change":"2023-06-27T12:42:07.470941+0000","last_active":"2023-06-27T12:42:07.439606+0000","last_peered":"2023-06-27T12:42:07.439606+0000","last_clean":"2023-06-27T12:42:07.439606+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:07.439606+0000","last_undegraded":"2023-06-27T12:42:07.439606+0000","last_fullsized":"2023-06-27T12:42:07.439606+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3697606,"num_objects":33,"num_object_clones":27,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":237,"num_read_kb":149309,"num_write":166,"num_write_kb":47587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"229'151","reported_seq":605,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-06-27T12:42:07.441180+0000","last_change":"2023-06-27T12:42:07.459059+0000","last_active":"2023-06-27T12:42:07.441180+0000","last_peered":"2023-06-27T12:42:07.441180+0000","last_clean":"2023-06-27T12:42:07.441180+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:07.441180+0000","last_undegraded":"2023-06-27T12:42:07.441180+0000","last_fullsized":"2023-06-27T12:42:07.441180+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5630477,"num_objects":18,"num_object_clones":15,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":109,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"199'60","reported_seq":612,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:08.485680+0000","last_change":"2023-06-27T12:42:08.487713+0000","last_active":"2023-06-27T12:42:08.485680+0000","last_peered":"2023-06-27T12:42:08.485680+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:08.485680+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916064,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":382,"reported_epoch":235,"state":"active+undersized","last_fresh":"2023-06-27T12:42:08.461330+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:08.461330+0000","last_peered":"2023-06-27T12:42:08.461330+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:08.461330+0000","last_undegraded":"2023-06-27T12:42:08.461330+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":534,"reported_epoch":235,"state":"active+undersized","last_fresh":"2023-06-27T12:42:08.510627+0000","last_change":"2023-06-27T12:42:08.528164+0000","last_active":"2023-06-27T12:42:08.510627+0000","last_peered":"2023-06-27T12:42:08.510627+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:08.510627+0000","last_undegraded":"2023-06-27T12:42:08.510627+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":363,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-06-27T12:42:07.439296+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:07.439296+0000","last_peered":"2023-06-27T12:42:07.439296+0000","last_clean":"2023-06-27T12:42:07.439296+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:07.439296+0000","last_undegraded":"2023-06-27T12:42:07.439296+0000","last_fullsized":"2023-06-27T12:42:07.439296+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"226'127","reported_seq":684,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:08.465491+0000","last_change":"2023-06-27T12:42:08.471483+0000","last_active":"2023-06-27T12:42:08.465491+0000","last_peered":"2023-06-27T12:42:08.465491+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:08.465491+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6722932,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":61,"num_read_kb":56879,"num_write":83,"num_write_kb":22643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":364,"reported_epoch":235,"state":"active+undersized","last_fresh":"2023-06-27T12:42:08.467058+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:08.467058+0000","last_peered":"2023-06-27T12:42:08.467058+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:08.467058+0000","last_undegraded":"2023-06-27T12:42:08.467058+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"225'316","reported_seq":311,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:08.467013+0000","last_change":"2023-06-27T12:42:08.476499+0000","last_active":"2023-06-27T12:42:08.467013+0000","last_peered":"2023-06-27T12:42:08.467013+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:08.467013+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":28,"num_read_kb":17194,"num_write":46,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":361,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-06-27T12:42:08.468172+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:08.468172+0000","last_peered":"2023-06-27T12:42:08.468172+0000","last_clean":"2023-06-27T12:42:08.468172+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:08.468172+0000","last_undegraded":"2023-06-27T12:42:08.468172+0000","last_fullsized":"2023-06-27T12:42:08.468172+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":115,"reported_epoch":234,"state":"active+undersized","last_fresh":"2023-06-27T12:42:07.443090+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:07.443090+0000","last_peered":"2023-06-27T12:42:07.443090+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:07.443090+0000","last_undegraded":"2023-06-27T12:42:07.443090+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"235'281","reported_seq":862,"reported_epoch":235,"state":"active+undersized+degraded+snaptrim","last_fresh":"2023-06-27T12:42:08.556533+0000","last_change":"2023-06-27T12:42:08.469650+0000","last_active":"2023-06-27T12:42:08.556533+0000","last_peered":"2023-06-27T12:42:08.556533+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:08.556533+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12588641,"num_objects":37,"num_object_clones":31,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":246,"num_read_kb":150312,"num_write":207,"num_write_kb":60840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":37}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":363,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-06-27T12:42:07.440313+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:07.440313+0000","last_peered":"2023-06-27T12:42:07.440313+0000","last_clean":"2023-06-27T12:42:07.440313+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:07.440313+0000","last_undegraded":"2023-06-27T12:42:07.440313+0000","last_fullsized":"2023-06-27T12:42:07.440313+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"220'143","reported_seq":274,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:08.523814+0000","last_change":"2023-06-27T12:42:08.550799+0000","last_active":"2023-06-27T12:42:08.523814+0000","last_peered":"2023-06-27T12:42:08.523814+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:08.523814+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117781,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":60,"num_read_kb":35684,"num_write":33,"num_write_kb":9435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":235,"state":"active+undersized","last_fresh":"2023-06-27T12:42:08.455936+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:08.455936+0000","last_peered":"2023-06-27T12:42:08.455936+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:08.455936+0000","last_undegraded":"2023-06-27T12:42:08.455936+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"230'135","reported_seq":710,"reported_epoch":235,"state":"active+undersized+degraded+snaptrim","last_fresh":"2023-06-27T12:42:08.453211+0000","last_change":"2023-06-27T12:42:08.453211+0000","last_active":"2023-06-27T12:42:08.453211+0000","last_peered":"2023-06-27T12:42:08.453211+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:08.453211+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9193594,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":82,"num_read_kb":89523,"num_write":96,"num_write_kb":28435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"218'93","reported_seq":269,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:08.484650+0000","last_change":"2023-06-27T12:42:08.488191+0000","last_active":"2023-06-27T12:42:08.484650+0000","last_peered":"2023-06-27T12:42:08.484650+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:08.484650+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2739607,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":50,"num_read_kb":26195,"num_write":26,"num_write_kb":7438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":382,"reported_epoch":235,"state":"active+undersized","last_fresh":"2023-06-27T12:42:08.457521+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:08.457521+0000","last_peered":"2023-06-27T12:42:08.457521+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:08.457521+0000","last_undegraded":"2023-06-27T12:42:08.457521+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":361,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-06-27T12:42:08.469595+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:08.469595+0000","last_peered":"2023-06-27T12:42:08.469595+0000","last_clean":"2023-06-27T12:42:08.469595+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:08.469595+0000","last_undegraded":"2023-06-27T12:42:08.469595+0000","last_fullsized":"2023-06-27T12:42:08.469595+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"231'163","reported_seq":700,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:08.462113+0000","last_change":"2023-06-27T12:42:08.464094+0000","last_active":"2023-06-27T12:42:08.462113+0000","last_peered":"2023-06-27T12:42:08.462113+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:08.462113+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857388,"num_objects":17,"num_object_clones":14,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":78,"num_read_kb":73813,"num_write":79,"num_write_kb":23224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":85734157,"num_objects":228,"num_object_clones":185,"num_object_copies":684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":1385,"num_read_kb":962059,"num_write":1191,"num_write_kb":346613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2243,"ondisk_log_size":2243,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673215,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":180804,"kb_used_data":149884,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94191036,"statfs":{"total":96636764160,"available":96451620864,"internally_reserved":0,"allocated":153481216,"data_stored":173305580,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":190772,"kb_used_data":159852,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94181068,"statfs":{"total":96636764160,"available":96441413632,"internally_reserved":0,"allocated":163688448,"data_stored":184716969,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673215,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":200396,"kb_used_data":169476,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94171444,"statfs":{"total":96636764160,"available":96431558656,"internally_reserved":0,"allocated":173543424,"data_stored":196400089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:12.471 INFO:tasks.rados.rados.0.smithi138.stdout:951: done (1 left) 2023-06-27T12:42:12.471 INFO:tasks.rados.rados.0.smithi138.stdout:952: write oid 47 current snap is 108 2023-06-27T12:42:12.471 INFO:tasks.rados.rados.0.smithi138.stdout:949: finishing write tid 1 to smithi138138488-48 2023-06-27T12:42:12.471 INFO:tasks.rados.rados.0.smithi138.stdout:952: seq_num 463 ranges {581641=621467,1994104=780490,3432022=349619} 2023-06-27T12:42:12.479 INFO:tasks.rados.rados.0.smithi138.stdout:952: writing smithi138138488-47 from 581641 to 1203108 tid 1 2023-06-27T12:42:12.490 INFO:tasks.rados.rados.0.smithi138.stdout:952: writing smithi138138488-47 from 1994104 to 2774594 tid 2 2023-06-27T12:42:12.497 INFO:tasks.rados.rados.0.smithi138.stdout:952: writing smithi138138488-47 from 3432022 to 3781641 tid 3 2023-06-27T12:42:12.497 INFO:tasks.rados.rados.0.smithi138.stdout:953: write (excl) oid 38 current snap is 108 2023-06-27T12:42:12.497 INFO:tasks.rados.rados.0.smithi138.stdout:949: finishing write tid 2 to smithi138138488-48 2023-06-27T12:42:12.498 INFO:tasks.rados.rados.0.smithi138.stdout:949: finishing write tid 3 to smithi138138488-48 2023-06-27T12:42:12.498 INFO:tasks.rados.rados.0.smithi138.stdout:953: seq_num 464 ranges {598448=413069,1768939=524148,2775798=581861,3781488=16960} 2023-06-27T12:42:12.504 INFO:tasks.rados.rados.0.smithi138.stdout:953: writing smithi138138488-38 from 598448 to 1011517 tid 1 2023-06-27T12:42:12.512 INFO:tasks.rados.rados.0.smithi138.stdout:953: writing smithi138138488-38 from 1768939 to 2293087 tid 2 2023-06-27T12:42:12.520 INFO:tasks.rados.rados.0.smithi138.stdout:953: writing smithi138138488-38 from 2775798 to 3357659 tid 3 2023-06-27T12:42:12.523 INFO:tasks.rados.rados.0.smithi138.stdout:953: writing smithi138138488-38 from 3781488 to 3798448 tid 4 2023-06-27T12:42:12.523 INFO:tasks.rados.rados.0.smithi138.stdout:954: read oid 31 snap -1 2023-06-27T12:42:12.523 INFO:tasks.rados.rados.0.smithi138.stdout:954: expect deleted 2023-06-27T12:42:12.523 INFO:tasks.rados.rados.0.smithi138.stdout:955: read oid 4 snap -1 2023-06-27T12:42:12.524 INFO:tasks.rados.rados.0.smithi138.stdout:955: expect (ObjNum 421 snap 99 seq_num 421) 2023-06-27T12:42:12.524 INFO:tasks.rados.rados.0.smithi138.stdout:956: read oid 1 snap -1 2023-06-27T12:42:12.524 INFO:tasks.rados.rados.0.smithi138.stdout:956: expect (ObjNum 406 snap 98 seq_num 406) 2023-06-27T12:42:12.524 INFO:tasks.rados.rados.0.smithi138.stdout:957: delete oid 11 current snap is 108 2023-06-27T12:42:12.524 INFO:tasks.rados.rados.0.smithi138.stdout:949: finishing write tid 4 to smithi138138488-48 2023-06-27T12:42:12.525 INFO:tasks.rados.rados.0.smithi138.stdout:949: finishing write tid 5 to smithi138138488-48 2023-06-27T12:42:12.525 INFO:tasks.rados.rados.0.smithi138.stdout:949: oid 48 updating version 0 to 168 2023-06-27T12:42:12.525 INFO:tasks.rados.rados.0.smithi138.stdout:949: oid 48 version 168 is already newer than 168 2023-06-27T12:42:12.525 INFO:tasks.rados.rados.0.smithi138.stdout:949: oid 48 version 168 is already newer than 168 2023-06-27T12:42:12.525 INFO:tasks.rados.rados.0.smithi138.stdout:949: oid 48 version 168 is already newer than 168 2023-06-27T12:42:12.525 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 168 (ObjNum 461 snap 108 seq_num 461) dirty exists 2023-06-27T12:42:12.526 INFO:tasks.rados.rados.0.smithi138.stdout:949: left oid 48 (ObjNum 461 snap 108 seq_num 461) 2023-06-27T12:42:12.589 INFO:tasks.rados.rados.0.smithi138.stdout:952: finishing write tid 1 to smithi138138488-47 2023-06-27T12:42:12.593 INFO:tasks.rados.rados.0.smithi138.stdout:953: finishing write tid 1 to smithi138138488-38 2023-06-27T12:42:12.625 INFO:tasks.rados.rados.0.smithi138.stdout:952: finishing write tid 2 to smithi138138488-47 2023-06-27T12:42:12.633 INFO:tasks.rados.rados.0.smithi138.stdout:952: finishing write tid 3 to smithi138138488-47 2023-06-27T12:42:12.634 INFO:tasks.rados.rados.0.smithi138.stdout:955: expect (ObjNum 421 snap 99 seq_num 421) 2023-06-27T12:42:12.838 INFO:tasks.rados.rados.0.smithi138.stdout:956: expect (ObjNum 406 snap 98 seq_num 406) 2023-06-27T12:42:12.985 INFO:tasks.rados.rados.0.smithi138.stdout:952: finishing write tid 4 to smithi138138488-47 2023-06-27T12:42:12.985 INFO:tasks.rados.rados.0.smithi138.stdout:952: finishing write tid 5 to smithi138138488-47 2023-06-27T12:42:12.985 INFO:tasks.rados.rados.0.smithi138.stdout:952: oid 47 updating version 0 to 145 2023-06-27T12:42:12.986 INFO:tasks.rados.rados.0.smithi138.stdout:952: oid 47 version 145 is already newer than 145 2023-06-27T12:42:12.986 INFO:tasks.rados.rados.0.smithi138.stdout:952: oid 47 version 145 is already newer than 145 2023-06-27T12:42:12.986 INFO:tasks.rados.rados.0.smithi138.stdout:952: oid 47 version 145 is already newer than 145 2023-06-27T12:42:12.987 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 47 v 145 (ObjNum 463 snap 108 seq_num 463) dirty exists 2023-06-27T12:42:12.987 INFO:tasks.rados.rados.0.smithi138.stdout:952: left oid 47 (ObjNum 463 snap 108 seq_num 463) 2023-06-27T12:42:12.987 INFO:tasks.rados.rados.0.smithi138.stdout:953: finishing write tid 2 to smithi138138488-38 2023-06-27T12:42:12.987 INFO:tasks.rados.rados.0.smithi138.stdout:953: finishing write tid 3 to smithi138138488-38 2023-06-27T12:42:12.988 INFO:tasks.rados.rados.0.smithi138.stdout:949: done (6 left) 2023-06-27T12:42:12.988 INFO:tasks.rados.rados.0.smithi138.stdout:952: done (5 left) 2023-06-27T12:42:12.988 INFO:tasks.rados.rados.0.smithi138.stdout:954: done (4 left) 2023-06-27T12:42:12.988 INFO:tasks.rados.rados.0.smithi138.stdout:955: done (3 left) 2023-06-27T12:42:12.989 INFO:tasks.rados.rados.0.smithi138.stdout:956: done (2 left) 2023-06-27T12:42:12.989 INFO:tasks.rados.rados.0.smithi138.stdout:957: done (1 left) 2023-06-27T12:42:12.989 INFO:tasks.rados.rados.0.smithi138.stdout:958: snap_create 2023-06-27T12:42:12.990 INFO:tasks.rados.rados.0.smithi138.stdout:953: finishing write tid 4 to smithi138138488-38 2023-06-27T12:42:12.990 INFO:tasks.rados.rados.0.smithi138.stdout:953: finishing write tid 5 to smithi138138488-38 2023-06-27T12:42:12.990 INFO:tasks.rados.rados.0.smithi138.stdout:953: finishing write tid 6 to smithi138138488-38 2023-06-27T12:42:12.990 INFO:tasks.rados.rados.0.smithi138.stdout:953: oid 38 updating version 0 to 23 2023-06-27T12:42:12.991 INFO:tasks.rados.rados.0.smithi138.stdout:953: oid 38 version 23 is already newer than 23 2023-06-27T12:42:12.991 INFO:tasks.rados.rados.0.smithi138.stdout:953: oid 38 version 23 is already newer than 23 2023-06-27T12:42:12.991 INFO:tasks.rados.rados.0.smithi138.stdout:953: oid 38 version 23 is already newer than 23 2023-06-27T12:42:12.992 INFO:tasks.rados.rados.0.smithi138.stdout:953: oid 38 version 23 is already newer than 19 2023-06-27T12:42:12.992 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 23 (ObjNum 464 snap 108 seq_num 464) dirty exists 2023-06-27T12:42:12.992 INFO:tasks.rados.rados.0.smithi138.stdout:953: left oid 38 (ObjNum 464 snap 108 seq_num 464) 2023-06-27T12:42:13.470 INFO:tasks.rados.rados.0.smithi138.stdout:953: done (1 left) 2023-06-27T12:42:13.470 INFO:tasks.rados.rados.0.smithi138.stdout:958: done (0 left) 2023-06-27T12:42:13.470 INFO:tasks.rados.rados.0.smithi138.stdout:959: snap_create 2023-06-27T12:42:13.707 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:13.840 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:13.835+0000 7f375bde5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:13.843 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:13.839+0000 7f375bde5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:13.888 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:13.891 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:13.892 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:14.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~99s 2023-06-27T12:42:14.477 INFO:tasks.rados.rados.0.smithi138.stdout:959: done (0 left) 2023-06-27T12:42:14.478 INFO:tasks.rados.rados.0.smithi138.stdout:960: write (excl) oid 44 current snap is 110 2023-06-27T12:42:14.478 INFO:tasks.rados.rados.0.smithi138.stdout:960: seq_num 466 ranges {632062=563567,1885903=527817,2830644=720758} 2023-06-27T12:42:14.484 INFO:tasks.rados.rados.0.smithi138.stdout:960: writing smithi138138488-44 from 632062 to 1195629 tid 1 2023-06-27T12:42:14.491 INFO:tasks.rados.rados.0.smithi138.stdout:960: writing smithi138138488-44 from 1885903 to 2413720 tid 2 2023-06-27T12:42:14.497 INFO:tasks.rados.rados.0.smithi138.stdout:960: writing smithi138138488-44 from 2830644 to 3551402 tid 3 2023-06-27T12:42:14.497 INFO:tasks.rados.rados.0.smithi138.stdout:961: delete oid 38 current snap is 110 2023-06-27T12:42:14.554 INFO:tasks.rados.rados.0.smithi138.stdout:961: done (1 left) 2023-06-27T12:42:14.554 INFO:tasks.rados.rados.0.smithi138.stdout:962: write (excl) oid 13 current snap is 110 2023-06-27T12:42:14.554 INFO:tasks.rados.rados.0.smithi138.stdout:962: seq_num 468 ranges {665676=714065,1886514=531486,3169137=696539} 2023-06-27T12:42:14.561 INFO:tasks.rados.rados.0.smithi138.stdout:962: writing smithi138138488-13 from 665676 to 1379741 tid 1 2023-06-27T12:42:14.567 INFO:tasks.rados.rados.0.smithi138.stdout:962: writing smithi138138488-13 from 1886514 to 2418000 tid 2 2023-06-27T12:42:14.576 INFO:tasks.rados.rados.0.smithi138.stdout:962: writing smithi138138488-13 from 3169137 to 3865676 tid 3 2023-06-27T12:42:14.576 INFO:tasks.rados.rados.0.smithi138.stdout:963: delete oid 17 current snap is 110 2023-06-27T12:42:14.576 INFO:tasks.rados.rados.0.smithi138.stdout:960: finishing write tid 1 to smithi138138488-44 2023-06-27T12:42:14.594 INFO:tasks.rados.rados.0.smithi138.stdout:960: finishing write tid 2 to smithi138138488-44 2023-06-27T12:42:14.615 INFO:tasks.rados.rados.0.smithi138.stdout:962: finishing write tid 1 to smithi138138488-13 2023-06-27T12:42:14.633 INFO:tasks.rados.rados.0.smithi138.stdout:963: done (2 left) 2023-06-27T12:42:14.633 INFO:tasks.rados.rados.0.smithi138.stdout:964: write oid 18 current snap is 110 2023-06-27T12:42:14.634 INFO:tasks.rados.rados.0.smithi138.stdout:964: seq_num 470 ranges {699290=464563,1603478=535155,2823983=765846} 2023-06-27T12:42:14.640 INFO:tasks.rados.rados.0.smithi138.stdout:964: writing smithi138138488-18 from 699290 to 1163853 tid 1 2023-06-27T12:42:14.645 INFO:tasks.rados.rados.0.smithi138.stdout:964: writing smithi138138488-18 from 1603478 to 2138633 tid 2 2023-06-27T12:42:14.656 INFO:tasks.rados.rados.0.smithi138.stdout:964: writing smithi138138488-18 from 2823983 to 3589829 tid 3 2023-06-27T12:42:14.656 INFO:tasks.rados.rados.0.smithi138.stdout:965: read oid 42 snap -1 2023-06-27T12:42:14.657 INFO:tasks.rados.rados.0.smithi138.stdout:965: expect (ObjNum 427 snap 100 seq_num 427) 2023-06-27T12:42:14.657 INFO:tasks.rados.rados.0.smithi138.stdout:962: finishing write tid 2 to smithi138138488-13 2023-06-27T12:42:14.657 INFO:tasks.rados.rados.0.smithi138.stdout:962: finishing write tid 3 to smithi138138488-13 2023-06-27T12:42:14.657 INFO:tasks.rados.rados.0.smithi138.stdout:962: finishing write tid 4 to smithi138138488-13 2023-06-27T12:42:14.657 INFO:tasks.rados.rados.0.smithi138.stdout:966: write (excl) oid 24 current snap is 110 2023-06-27T12:42:14.658 INFO:tasks.rados.rados.0.smithi138.stdout:962: finishing write tid 5 to smithi138138488-13 2023-06-27T12:42:14.658 INFO:tasks.rados.rados.0.smithi138.stdout:962: oid 13 updating version 0 to 322 2023-06-27T12:42:14.658 INFO:tasks.rados.rados.0.smithi138.stdout:962: oid 13 version 322 is already newer than 319 2023-06-27T12:42:14.658 INFO:tasks.rados.rados.0.smithi138.stdout:962: oid 13 version 322 is already newer than 322 2023-06-27T12:42:14.658 INFO:tasks.rados.rados.0.smithi138.stdout:962: oid 13 version 322 is already newer than 322 2023-06-27T12:42:14.658 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 322 (ObjNum 468 snap 110 seq_num 468) dirty exists 2023-06-27T12:42:14.658 INFO:tasks.rados.rados.0.smithi138.stdout:962: left oid 13 (ObjNum 468 snap 110 seq_num 468) 2023-06-27T12:42:14.658 INFO:tasks.rados.rados.0.smithi138.stdout:960: finishing write tid 3 to smithi138138488-44 2023-06-27T12:42:14.659 INFO:tasks.rados.rados.0.smithi138.stdout:960: finishing write tid 4 to smithi138138488-44 2023-06-27T12:42:14.659 INFO:tasks.rados.rados.0.smithi138.stdout:966: seq_num 471 ranges {716097=539812,1945607=795166,3535053=381044} 2023-06-27T12:42:14.663 INFO:tasks.rados.rados.0.smithi138.stdout:966: writing smithi138138488-24 from 716097 to 1255909 tid 1 2023-06-27T12:42:14.672 INFO:tasks.rados.rados.0.smithi138.stdout:966: writing smithi138138488-24 from 1945607 to 2740773 tid 2 2023-06-27T12:42:14.680 INFO:tasks.rados.rados.0.smithi138.stdout:966: writing smithi138138488-24 from 3535053 to 3916097 tid 3 2023-06-27T12:42:14.680 INFO:tasks.rados.rados.0.smithi138.stdout:962: done (4 left) 2023-06-27T12:42:14.680 INFO:tasks.rados.rados.0.smithi138.stdout:967: write oid 20 current snap is 110 2023-06-27T12:42:14.681 INFO:tasks.rados.rados.0.smithi138.stdout:967: seq_num 472 ranges {732904=731414,2120442=538824,3278829=504743} 2023-06-27T12:42:14.689 INFO:tasks.rados.rados.0.smithi138.stdout:967: writing smithi138138488-20 from 732904 to 1464318 tid 1 2023-06-27T12:42:14.697 INFO:tasks.rados.rados.0.smithi138.stdout:967: writing smithi138138488-20 from 2120442 to 2659266 tid 2 2023-06-27T12:42:14.704 INFO:tasks.rados.rados.0.smithi138.stdout:967: writing smithi138138488-20 from 3278829 to 3783572 tid 3 2023-06-27T12:42:14.704 INFO:tasks.rados.rados.0.smithi138.stdout:968: snap_create 2023-06-27T12:42:14.704 INFO:tasks.rados.rados.0.smithi138.stdout:960: finishing write tid 5 to smithi138138488-44 2023-06-27T12:42:14.704 INFO:tasks.rados.rados.0.smithi138.stdout:960: oid 44 updating version 0 to 298 2023-06-27T12:42:14.705 INFO:tasks.rados.rados.0.smithi138.stdout:960: oid 44 version 298 is already newer than 298 2023-06-27T12:42:14.705 INFO:tasks.rados.rados.0.smithi138.stdout:960: oid 44 version 298 is already newer than 294 2023-06-27T12:42:14.705 INFO:tasks.rados.rados.0.smithi138.stdout:960: oid 44 version 298 is already newer than 298 2023-06-27T12:42:14.705 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 298 (ObjNum 466 snap 110 seq_num 466) dirty exists 2023-06-27T12:42:14.705 INFO:tasks.rados.rados.0.smithi138.stdout:960: left oid 44 (ObjNum 466 snap 110 seq_num 466) 2023-06-27T12:42:14.705 INFO:tasks.rados.rados.0.smithi138.stdout:965: expect (ObjNum 427 snap 100 seq_num 427) 2023-06-27T12:42:14.873 INFO:tasks.rados.rados.0.smithi138.stdout:966: finishing write tid 1 to smithi138138488-24 2023-06-27T12:42:14.874 INFO:tasks.rados.rados.0.smithi138.stdout:964: finishing write tid 1 to smithi138138488-18 2023-06-27T12:42:14.874 INFO:tasks.rados.rados.0.smithi138.stdout:964: finishing write tid 2 to smithi138138488-18 2023-06-27T12:42:14.874 INFO:tasks.rados.rados.0.smithi138.stdout:964: finishing write tid 3 to smithi138138488-18 2023-06-27T12:42:14.874 INFO:tasks.rados.rados.0.smithi138.stdout:964: finishing write tid 4 to smithi138138488-18 2023-06-27T12:42:14.874 INFO:tasks.rados.rados.0.smithi138.stdout:964: finishing write tid 5 to smithi138138488-18 2023-06-27T12:42:14.875 INFO:tasks.rados.rados.0.smithi138.stdout:964: oid 18 updating version 0 to 174 2023-06-27T12:42:14.875 INFO:tasks.rados.rados.0.smithi138.stdout:964: oid 18 version 174 is already newer than 174 2023-06-27T12:42:14.875 INFO:tasks.rados.rados.0.smithi138.stdout:964: oid 18 version 174 is already newer than 174 2023-06-27T12:42:14.875 INFO:tasks.rados.rados.0.smithi138.stdout:964: oid 18 version 174 is already newer than 174 2023-06-27T12:42:14.875 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 174 (ObjNum 470 snap 110 seq_num 470) dirty exists 2023-06-27T12:42:14.875 INFO:tasks.rados.rados.0.smithi138.stdout:964: left oid 18 (ObjNum 470 snap 110 seq_num 470) 2023-06-27T12:42:14.875 INFO:tasks.rados.rados.0.smithi138.stdout:967: finishing write tid 1 to smithi138138488-20 2023-06-27T12:42:14.876 INFO:tasks.rados.rados.0.smithi138.stdout:967: finishing write tid 2 to smithi138138488-20 2023-06-27T12:42:14.876 INFO:tasks.rados.rados.0.smithi138.stdout:967: finishing write tid 3 to smithi138138488-20 2023-06-27T12:42:14.876 INFO:tasks.rados.rados.0.smithi138.stdout:967: finishing write tid 4 to smithi138138488-20 2023-06-27T12:42:14.876 INFO:tasks.rados.rados.0.smithi138.stdout:967: finishing write tid 5 to smithi138138488-20 2023-06-27T12:42:14.876 INFO:tasks.rados.rados.0.smithi138.stdout:967: oid 20 updating version 0 to 151 2023-06-27T12:42:14.876 INFO:tasks.rados.rados.0.smithi138.stdout:967: oid 20 version 151 is already newer than 151 2023-06-27T12:42:14.876 INFO:tasks.rados.rados.0.smithi138.stdout:967: oid 20 version 151 is already newer than 151 2023-06-27T12:42:14.877 INFO:tasks.rados.rados.0.smithi138.stdout:967: oid 20 version 151 is already newer than 151 2023-06-27T12:42:14.877 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 151 (ObjNum 472 snap 110 seq_num 472) dirty exists 2023-06-27T12:42:14.877 INFO:tasks.rados.rados.0.smithi138.stdout:967: left oid 20 (ObjNum 472 snap 110 seq_num 472) 2023-06-27T12:42:14.877 INFO:tasks.rados.rados.0.smithi138.stdout:966: finishing write tid 2 to smithi138138488-24 2023-06-27T12:42:14.877 INFO:tasks.rados.rados.0.smithi138.stdout:966: finishing write tid 3 to smithi138138488-24 2023-06-27T12:42:14.877 INFO:tasks.rados.rados.0.smithi138.stdout:966: finishing write tid 4 to smithi138138488-24 2023-06-27T12:42:14.877 INFO:tasks.rados.rados.0.smithi138.stdout:966: finishing write tid 5 to smithi138138488-24 2023-06-27T12:42:14.878 INFO:tasks.rados.rados.0.smithi138.stdout:966: oid 24 updating version 0 to 302 2023-06-27T12:42:14.878 INFO:tasks.rados.rados.0.smithi138.stdout:966: oid 24 version 302 is already newer than 299 2023-06-27T12:42:14.878 INFO:tasks.rados.rados.0.smithi138.stdout:966: oid 24 version 302 is already newer than 302 2023-06-27T12:42:14.878 INFO:tasks.rados.rados.0.smithi138.stdout:966: oid 24 version 302 is already newer than 302 2023-06-27T12:42:14.878 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 302 (ObjNum 471 snap 110 seq_num 471) dirty exists 2023-06-27T12:42:14.879 INFO:tasks.rados.rados.0.smithi138.stdout:966: left oid 24 (ObjNum 471 snap 110 seq_num 471) 2023-06-27T12:42:15.427 INFO:tasks.rados.rados.0.smithi138.stdout:960: done (5 left) 2023-06-27T12:42:15.427 INFO:tasks.rados.rados.0.smithi138.stdout:964: done (4 left) 2023-06-27T12:42:15.427 INFO:tasks.rados.rados.0.smithi138.stdout:965: done (3 left) 2023-06-27T12:42:15.428 INFO:tasks.rados.rados.0.smithi138.stdout:966: done (2 left) 2023-06-27T12:42:15.428 INFO:tasks.rados.rados.0.smithi138.stdout:967: done (1 left) 2023-06-27T12:42:15.428 INFO:tasks.rados.rados.0.smithi138.stdout:968: done (0 left) 2023-06-27T12:42:15.428 INFO:tasks.rados.rados.0.smithi138.stdout:969: write oid 46 current snap is 111 2023-06-27T12:42:15.428 INFO:tasks.rados.rados.0.smithi138.stdout:969: seq_num 473 ranges {749711=406663,1778924=682482,3189899=716015} 2023-06-27T12:42:15.433 INFO:tasks.rados.rados.0.smithi138.stdout:969: writing smithi138138488-46 from 749711 to 1156374 tid 1 2023-06-27T12:42:15.440 INFO:tasks.rados.rados.0.smithi138.stdout:969: writing smithi138138488-46 from 1778924 to 2461406 tid 2 2023-06-27T12:42:15.447 INFO:tasks.rados.rados.0.smithi138.stdout:969: writing smithi138138488-46 from 3189899 to 3905914 tid 3 2023-06-27T12:42:15.448 INFO:tasks.rados.rados.0.smithi138.stdout:970: read oid 45 snap 109 2023-06-27T12:42:15.448 INFO:tasks.rados.rados.0.smithi138.stdout:970: expect deleted 2023-06-27T12:42:15.448 INFO:tasks.rados.rados.0.smithi138.stdout:971: snap_create 2023-06-27T12:42:15.478 INFO:tasks.rados.rados.0.smithi138.stdout:969: finishing write tid 1 to smithi138138488-46 2023-06-27T12:42:15.481 INFO:tasks.rados.rados.0.smithi138.stdout:969: finishing write tid 2 to smithi138138488-46 2023-06-27T12:42:15.485 INFO:tasks.rados.rados.0.smithi138.stdout:969: finishing write tid 3 to smithi138138488-46 2023-06-27T12:42:15.488 INFO:tasks.rados.rados.0.smithi138.stdout:969: finishing write tid 4 to smithi138138488-46 2023-06-27T12:42:15.489 INFO:tasks.rados.rados.0.smithi138.stdout:969: finishing write tid 5 to smithi138138488-46 2023-06-27T12:42:15.489 INFO:tasks.rados.rados.0.smithi138.stdout:969: oid 46 updating version 0 to 132 2023-06-27T12:42:15.489 INFO:tasks.rados.rados.0.smithi138.stdout:969: oid 46 version 132 is already newer than 132 2023-06-27T12:42:15.489 INFO:tasks.rados.rados.0.smithi138.stdout:969: oid 46 version 132 is already newer than 132 2023-06-27T12:42:15.490 INFO:tasks.rados.rados.0.smithi138.stdout:969: oid 46 version 132 is already newer than 132 2023-06-27T12:42:15.490 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 132 (ObjNum 473 snap 111 seq_num 473) dirty exists 2023-06-27T12:42:15.490 INFO:tasks.rados.rados.0.smithi138.stdout:969: left oid 46 (ObjNum 473 snap 111 seq_num 473) 2023-06-27T12:42:15.893 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:16.019 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:16.013+0000 7fdef5df6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:16.021 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:16.018+0000 7fdef5df6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:16.067 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:16.070 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:16.070 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:16.434 INFO:tasks.rados.rados.0.smithi138.stdout:969: done (2 left) 2023-06-27T12:42:16.434 INFO:tasks.rados.rados.0.smithi138.stdout:970: done (1 left) 2023-06-27T12:42:16.434 INFO:tasks.rados.rados.0.smithi138.stdout:971: done (0 left) 2023-06-27T12:42:16.434 INFO:tasks.rados.rados.0.smithi138.stdout:972: read oid 26 snap 109 2023-06-27T12:42:16.435 INFO:tasks.rados.rados.0.smithi138.stdout:972: expect (ObjNum 451 snap 104 seq_num 451) 2023-06-27T12:42:16.435 INFO:tasks.rados.rados.0.smithi138.stdout:973: snap_remove snap 111 2023-06-27T12:42:16.757 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:42:16.757 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-06-27T12:42:16.891 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:16.881+0000 7efc110df700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:16.893 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:16.890+0000 7efc110df700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:17.433 INFO:tasks.rados.rados.0.smithi138.stdout:973: done (1 left) 2023-06-27T12:42:17.434 INFO:tasks.rados.rados.0.smithi138.stdout:974: delete oid 28 current snap is 112 2023-06-27T12:42:17.434 INFO:tasks.rados.rados.0.smithi138.stdout:972: expect (ObjNum 451 snap 104 seq_num 451) 2023-06-27T12:42:17.436 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 1 (10000) 2023-06-27T12:42:17.454 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-06-27T12:42:17.585 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:17.576+0000 7f5d9b079700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:17.588 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:17.585+0000 7f5d9b079700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:17.637 INFO:tasks.rados.rados.0.smithi138.stdout:972: done (1 left) 2023-06-27T12:42:17.637 INFO:tasks.rados.rados.0.smithi138.stdout:974: done (0 left) 2023-06-27T12:42:17.637 INFO:tasks.rados.rados.0.smithi138.stdout:975: snap_remove snap 110 2023-06-27T12:42:18.071 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:18.194+0000 7f100749f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:18.202 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:18.198+0000 7f100749f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:18.247 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:18.250 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:18.250 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:18.501 INFO:tasks.rados.rados.0.smithi138.stdout:975: done (0 left) 2023-06-27T12:42:18.501 INFO:tasks.rados.rados.0.smithi138.stdout:976: delete oid 19 current snap is 112 2023-06-27T12:42:18.504 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.1 to 1 (10000) 2023-06-27T12:42:18.522 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-06-27T12:42:18.552 INFO:tasks.rados.rados.0.smithi138.stdout:976: done (0 left) 2023-06-27T12:42:18.553 INFO:tasks.rados.rados.0.smithi138.stdout:977: snap_remove snap 109 2023-06-27T12:42:18.653 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:18.643+0000 7f1cd424e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:18.656 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:18.652+0000 7f1cd424e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:977: done (0 left) 2023-06-27T12:42:19.507 INFO:tasks.rados.rados.0.smithi138.stdout:978: snap_remove snap 108 2023-06-27T12:42:19.511 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.2 to 1 (10000) 2023-06-27T12:42:19.529 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-06-27T12:42:19.698 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:19.690+0000 7fe17dbb8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:19.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:19.697+0000 7fe17dbb8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:19.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~105s 2023-06-27T12:42:20.252 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:20.379 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:20.375+0000 7f7d67ad9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:20.382 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:20.378+0000 7f7d67ad9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:20.427 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:20.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:20.431 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:20.509 INFO:tasks.rados.rados.0.smithi138.stdout:978: done (0 left) 2023-06-27T12:42:20.509 INFO:tasks.rados.rados.0.smithi138.stdout:979: delete oid 46 current snap is 112 2023-06-27T12:42:20.511 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.3 to 1 (10000) 2023-06-27T12:42:20.525 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-06-27T12:42:20.525 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:20.665 INFO:tasks.rados.rados.0.smithi138.stdout:979: done (0 left) 2023-06-27T12:42:20.665 INFO:tasks.rados.rados.0.smithi138.stdout:980: write (excl) oid 37 current snap is 112 2023-06-27T12:42:20.665 INFO:tasks.rados.rados.0.smithi138.stdout:980: seq_num 477 ranges {416939=707659,1896499=120440} 2023-06-27T12:42:20.665 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:20.658+0000 7f9cbc054700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:20.668 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:20.665+0000 7f9cbc054700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:20.672 INFO:tasks.rados.rados.0.smithi138.stdout:980: writing smithi138138488-37 from 416939 to 1124598 tid 1 2023-06-27T12:42:20.679 INFO:tasks.rados.rados.0.smithi138.stdout:980: writing smithi138138488-37 from 1896499 to 2016939 tid 2 2023-06-27T12:42:20.679 INFO:tasks.rados.rados.0.smithi138.stdout:981: read oid 29 snap -1 2023-06-27T12:42:20.679 INFO:tasks.rados.rados.0.smithi138.stdout:981: expect deleted 2023-06-27T12:42:20.679 INFO:tasks.rados.rados.0.smithi138.stdout:982: read oid 10 snap -1 2023-06-27T12:42:20.679 INFO:tasks.rados.rados.0.smithi138.stdout:982: expect deleted 2023-06-27T12:42:20.679 INFO:tasks.rados.rados.0.smithi138.stdout:983: read oid 14 snap -1 2023-06-27T12:42:20.680 INFO:tasks.rados.rados.0.smithi138.stdout:983: expect (ObjNum 218 snap 49 seq_num 218) 2023-06-27T12:42:20.680 INFO:tasks.rados.rados.0.smithi138.stdout:984: write (excl) oid 34 current snap is 112 2023-06-27T12:42:20.680 INFO:tasks.rados.rados.0.smithi138.stdout:984: seq_num 478 ranges {433746=782908,1954981=78765} 2023-06-27T12:42:20.686 INFO:tasks.rados.rados.0.smithi138.stdout:984: writing smithi138138488-34 from 433746 to 1216654 tid 1 2023-06-27T12:42:20.690 INFO:tasks.rados.rados.0.smithi138.stdout:984: writing smithi138138488-34 from 1954981 to 2033746 tid 2 2023-06-27T12:42:20.691 INFO:tasks.rados.rados.0.smithi138.stdout:985: read oid 47 snap -1 2023-06-27T12:42:20.691 INFO:tasks.rados.rados.0.smithi138.stdout:985: expect (ObjNum 463 snap 108 seq_num 463) 2023-06-27T12:42:20.691 INFO:tasks.rados.rados.0.smithi138.stdout:986: write (excl) oid 20 current snap is 112 2023-06-27T12:42:20.691 INFO:tasks.rados.rados.0.smithi138.stdout:986: seq_num 479 ranges {450553=574510,1613463=437090} 2023-06-27T12:42:20.696 INFO:tasks.rados.rados.0.smithi138.stdout:986: writing smithi138138488-20 from 450553 to 1025063 tid 1 2023-06-27T12:42:20.703 INFO:tasks.rados.rados.0.smithi138.stdout:986: writing smithi138138488-20 from 1613463 to 2050553 tid 2 2023-06-27T12:42:20.703 INFO:tasks.rados.rados.0.smithi138.stdout:987: read oid 8 snap -1 2023-06-27T12:42:20.703 INFO:tasks.rados.rados.0.smithi138.stdout:987: expect (ObjNum 457 snap 105 seq_num 457) 2023-06-27T12:42:20.703 INFO:tasks.rados.rados.0.smithi138.stdout:988: write oid 45 current snap is 112 2023-06-27T12:42:20.704 INFO:tasks.rados.rados.0.smithi138.stdout:988: seq_num 480 ranges {467360=649759,1671945=395415} 2023-06-27T12:42:20.710 INFO:tasks.rados.rados.0.smithi138.stdout:988: writing smithi138138488-45 from 467360 to 1117119 tid 1 2023-06-27T12:42:20.717 INFO:tasks.rados.rados.0.smithi138.stdout:988: writing smithi138138488-45 from 1671945 to 2067360 tid 2 2023-06-27T12:42:20.717 INFO:tasks.rados.rados.0.smithi138.stdout:989: delete oid 21 current snap is 112 2023-06-27T12:42:20.726 INFO:tasks.rados.rados.0.smithi138.stdout:983: expect (ObjNum 218 snap 49 seq_num 218) 2023-06-27T12:42:20.895 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:20.897 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:20.908 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2023-06-27T12:42:20.503781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":86137492,"num_objects":237,"num_object_clones":193,"num_object_copies":711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1424,"num_read_kb":988636,"num_write":1237,"num_write_kb":362236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2399,"ondisk_log_size":2399,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":770580,"kb_used_data":646900,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376716780,"statfs":{"total":386547056640,"available":385757982720,"internally_reserved":0,"allocated":662425600,"data_stored":751117448,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6000165,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":20,"num_read_kb":13427,"num_write":30,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.075049"},"pg_stats":[{"pgid":"3.f","version":"241'25","reported_seq":583,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.518431+0000","last_change":"2023-06-27T12:42:18.523085+0000","last_active":"2023-06-27T12:42:18.518431+0000","last_peered":"2023-06-27T12:42:18.518431+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:18.518431+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":8,"num_read_kb":4,"num_write":19,"num_write_kb":5014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"241'151","reported_seq":761,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.442699+0000","last_change":"2023-06-27T12:42:12.511488+0000","last_active":"2023-06-27T12:42:16.442699+0000","last_peered":"2023-06-27T12:42:16.442699+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:16.442699+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11311309,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":103,"num_read_kb":107986,"num_write":106,"num_write_kb":31310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"241'174","reported_seq":751,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.548528+0000","last_change":"2023-06-27T12:42:18.553261+0000","last_active":"2023-06-27T12:42:18.548528+0000","last_peered":"2023-06-27T12:42:18.548528+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:18.548528+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9294436,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":127,"num_read_kb":61154,"num_write":121,"num_write_kb":35887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"230'100","reported_seq":716,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.517985+0000","last_change":"2023-06-27T12:42:18.529241+0000","last_active":"2023-06-27T12:42:18.517985+0000","last_peered":"2023-06-27T12:42:18.517985+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:18.517985+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9865907,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":96,"num_read_kb":54526,"num_write":79,"num_write_kb":22429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"231'59","reported_seq":282,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.503221+0000","last_change":"2023-06-27T12:42:18.523206+0000","last_active":"2023-06-27T12:42:18.503221+0000","last_peered":"2023-06-27T12:42:18.503221+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:18.503221+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4722833,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":24677,"num_write":29,"num_write_kb":8919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"241'302","reported_seq":779,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.516330+0000","last_change":"2023-06-27T12:42:18.525961+0000","last_active":"2023-06-27T12:42:18.516330+0000","last_peered":"2023-06-27T12:42:18.516330+0000","last_clean":"2023-06-27T12:42:18.516330+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:18.516330+0000","last_undegraded":"2023-06-27T12:42:18.516330+0000","last_fullsized":"2023-06-27T12:42:18.516330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7748159,"num_objects":34,"num_object_clones":28,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":241,"num_read_kb":149311,"num_write":176,"num_write_kb":51036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"244'153","reported_seq":633,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.532107+0000","last_change":"2023-06-27T12:42:18.532233+0000","last_active":"2023-06-27T12:42:18.532107+0000","last_peered":"2023-06-27T12:42:18.532107+0000","last_clean":"2023-06-27T12:42:18.532107+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:18.532107+0000","last_undegraded":"2023-06-27T12:42:18.532107+0000","last_fullsized":"2023-06-27T12:42:18.532107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2201783,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":109,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"238'62","reported_seq":634,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.524049+0000","last_change":"2023-06-27T12:42:18.527029+0000","last_active":"2023-06-27T12:42:18.524049+0000","last_peered":"2023-06-27T12:42:18.524049+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:18.524049+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":55,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":398,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-06-27T12:42:16.448959+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:16.448959+0000","last_peered":"2023-06-27T12:42:16.448959+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:16.448959+0000","last_undegraded":"2023-06-27T12:42:16.448959+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":555,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-06-27T12:42:18.548998+0000","last_change":"2023-06-27T12:42:18.552806+0000","last_active":"2023-06-27T12:42:18.548998+0000","last_peered":"2023-06-27T12:42:18.548998+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:18.548998+0000","last_undegraded":"2023-06-27T12:42:18.548998+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":383,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.516764+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:18.516764+0000","last_peered":"2023-06-27T12:42:18.516764+0000","last_clean":"2023-06-27T12:42:18.516764+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:18.516764+0000","last_undegraded":"2023-06-27T12:42:18.516764+0000","last_fullsized":"2023-06-27T12:42:18.516764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"242'132","reported_seq":708,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.447002+0000","last_change":"2023-06-27T12:42:12.514010+0000","last_active":"2023-06-27T12:42:16.447002+0000","last_peered":"2023-06-27T12:42:16.447002+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:16.447002+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260756,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":62,"num_read_kb":56880,"num_write":88,"num_write_kb":24408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":380,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-06-27T12:42:18.541748+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:18.541748+0000","last_peered":"2023-06-27T12:42:18.541748+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:18.541748+0000","last_undegraded":"2023-06-27T12:42:18.541748+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"241'322","reported_seq":335,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.448048+0000","last_change":"2023-06-27T12:42:12.515191+0000","last_active":"2023-06-27T12:42:16.448048+0000","last_peered":"2023-06-27T12:42:16.448048+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:16.448048+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3865676,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":17195,"num_write":51,"num_write_kb":15714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":377,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.525853+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:18.525853+0000","last_peered":"2023-06-27T12:42:18.525853+0000","last_clean":"2023-06-27T12:42:18.525853+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:18.525853+0000","last_undegraded":"2023-06-27T12:42:18.525853+0000","last_fullsized":"2023-06-27T12:42:18.525853+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":135,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-06-27T12:42:18.522836+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:18.522836+0000","last_peered":"2023-06-27T12:42:18.522836+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:18.522836+0000","last_undegraded":"2023-06-27T12:42:18.522836+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"238'283","reported_seq":892,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.544690+0000","last_change":"2023-06-27T12:42:18.544690+0000","last_active":"2023-06-27T12:42:18.544690+0000","last_peered":"2023-06-27T12:42:18.544690+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:18.544690+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874228,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":254,"num_read_kb":157621,"num_write":207,"num_write_kb":60840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":383,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.511431+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:18.511431+0000","last_peered":"2023-06-27T12:42:18.511431+0000","last_clean":"2023-06-27T12:42:18.511431+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:18.511431+0000","last_undegraded":"2023-06-27T12:42:18.511431+0000","last_fullsized":"2023-06-27T12:42:18.511431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"220'143","reported_seq":298,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.549869+0000","last_change":"2023-06-27T12:42:18.553600+0000","last_active":"2023-06-27T12:42:18.549869+0000","last_peered":"2023-06-27T12:42:18.549869+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:18.549869+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117781,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":67,"num_read_kb":41522,"num_write":33,"num_write_kb":9435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-06-27T12:42:16.448552+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:16.448552+0000","last_peered":"2023-06-27T12:42:16.448552+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:16.448552+0000","last_undegraded":"2023-06-27T12:42:16.448552+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"230'135","reported_seq":733,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.463610+0000","last_change":"2023-06-27T12:42:12.506312+0000","last_active":"2023-06-27T12:42:16.463610+0000","last_peered":"2023-06-27T12:42:16.463610+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:16.463610+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9193594,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":84,"num_read_kb":96517,"num_write":96,"num_write_kb":28435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"218'93","reported_seq":290,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.542947+0000","last_change":"2023-06-27T12:42:18.545250+0000","last_active":"2023-06-27T12:42:18.542947+0000","last_peered":"2023-06-27T12:42:18.542947+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:18.542947+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2739607,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":50,"num_read_kb":26195,"num_write":26,"num_write_kb":7438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":398,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-06-27T12:42:16.449553+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:16.449553+0000","last_peered":"2023-06-27T12:42:16.449553+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:16.449553+0000","last_undegraded":"2023-06-27T12:42:16.449553+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":377,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.538808+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:18.538808+0000","last_peered":"2023-06-27T12:42:18.538808+0000","last_clean":"2023-06-27T12:42:18.538808+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:18.538808+0000","last_undegraded":"2023-06-27T12:42:18.538808+0000","last_fullsized":"2023-06-27T12:42:18.538808+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"238'168","reported_seq":724,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.447503+0000","last_change":"2023-06-27T12:42:12.552329+0000","last_active":"2023-06-27T12:42:16.447503+0000","last_peered":"2023-06-27T12:42:16.447503+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:16.447503+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6941423,"num_objects":18,"num_object_clones":15,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":79,"num_read_kb":73814,"num_write":84,"num_write_kb":25060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":86137492,"num_objects":237,"num_object_clones":193,"num_object_copies":711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1424,"num_read_kb":988636,"num_write":1237,"num_write_kb":362236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2302,"ondisk_log_size":2302,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":187176,"kb_used_data":156256,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94184664,"statfs":{"total":96636764160,"available":96445095936,"internally_reserved":0,"allocated":160006144,"data_stored":181218304,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":195236,"kb_used_data":164316,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94176604,"statfs":{"total":96636764160,"available":96436842496,"internally_reserved":0,"allocated":168259584,"data_stored":189669412,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":206888,"kb_used_data":175968,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94164952,"statfs":{"total":96636764160,"available":96424910848,"internally_reserved":0,"allocated":180191232,"data_stored":203190180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:20.911 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:21.022 INFO:tasks.rados.rados.0.smithi138.stdout:985: expect (ObjNum 463 snap 108 seq_num 463) 2023-06-27T12:42:21.038 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:21.031+0000 7f06d3e47700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:21.041 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:21.038+0000 7f06d3e47700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:21.212 INFO:tasks.rados.rados.0.smithi138.stdout:987: expect (ObjNum 457 snap 105 seq_num 457) 2023-06-27T12:42:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:21.239 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:21.250 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2023-06-27T12:42:20.503781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":86137492,"num_objects":237,"num_object_clones":193,"num_object_copies":711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1424,"num_read_kb":988636,"num_write":1237,"num_write_kb":362236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2399,"ondisk_log_size":2399,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":770580,"kb_used_data":646900,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376716780,"statfs":{"total":386547056640,"available":385757982720,"internally_reserved":0,"allocated":662425600,"data_stored":751117448,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6000165,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":20,"num_read_kb":13427,"num_write":30,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.075049"},"pg_stats":[{"pgid":"3.f","version":"241'25","reported_seq":583,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.518431+0000","last_change":"2023-06-27T12:42:18.523085+0000","last_active":"2023-06-27T12:42:18.518431+0000","last_peered":"2023-06-27T12:42:18.518431+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:18.518431+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":8,"num_read_kb":4,"num_write":19,"num_write_kb":5014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"241'151","reported_seq":761,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.442699+0000","last_change":"2023-06-27T12:42:12.511488+0000","last_active":"2023-06-27T12:42:16.442699+0000","last_peered":"2023-06-27T12:42:16.442699+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:16.442699+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11311309,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":103,"num_read_kb":107986,"num_write":106,"num_write_kb":31310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"241'174","reported_seq":751,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.548528+0000","last_change":"2023-06-27T12:42:18.553261+0000","last_active":"2023-06-27T12:42:18.548528+0000","last_peered":"2023-06-27T12:42:18.548528+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:18.548528+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9294436,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":127,"num_read_kb":61154,"num_write":121,"num_write_kb":35887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"230'100","reported_seq":716,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.517985+0000","last_change":"2023-06-27T12:42:18.529241+0000","last_active":"2023-06-27T12:42:18.517985+0000","last_peered":"2023-06-27T12:42:18.517985+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:18.517985+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9865907,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":96,"num_read_kb":54526,"num_write":79,"num_write_kb":22429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"231'59","reported_seq":282,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.503221+0000","last_change":"2023-06-27T12:42:18.523206+0000","last_active":"2023-06-27T12:42:18.503221+0000","last_peered":"2023-06-27T12:42:18.503221+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:18.503221+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4722833,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":24677,"num_write":29,"num_write_kb":8919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"241'302","reported_seq":779,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.516330+0000","last_change":"2023-06-27T12:42:18.525961+0000","last_active":"2023-06-27T12:42:18.516330+0000","last_peered":"2023-06-27T12:42:18.516330+0000","last_clean":"2023-06-27T12:42:18.516330+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:18.516330+0000","last_undegraded":"2023-06-27T12:42:18.516330+0000","last_fullsized":"2023-06-27T12:42:18.516330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7748159,"num_objects":34,"num_object_clones":28,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":241,"num_read_kb":149311,"num_write":176,"num_write_kb":51036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"244'153","reported_seq":633,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.532107+0000","last_change":"2023-06-27T12:42:18.532233+0000","last_active":"2023-06-27T12:42:18.532107+0000","last_peered":"2023-06-27T12:42:18.532107+0000","last_clean":"2023-06-27T12:42:18.532107+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:18.532107+0000","last_undegraded":"2023-06-27T12:42:18.532107+0000","last_fullsized":"2023-06-27T12:42:18.532107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2201783,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":109,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"238'62","reported_seq":634,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.524049+0000","last_change":"2023-06-27T12:42:18.527029+0000","last_active":"2023-06-27T12:42:18.524049+0000","last_peered":"2023-06-27T12:42:18.524049+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:18.524049+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":55,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":398,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-06-27T12:42:16.448959+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:16.448959+0000","last_peered":"2023-06-27T12:42:16.448959+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:16.448959+0000","last_undegraded":"2023-06-27T12:42:16.448959+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":555,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-06-27T12:42:18.548998+0000","last_change":"2023-06-27T12:42:18.552806+0000","last_active":"2023-06-27T12:42:18.548998+0000","last_peered":"2023-06-27T12:42:18.548998+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:18.548998+0000","last_undegraded":"2023-06-27T12:42:18.548998+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":383,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.516764+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:18.516764+0000","last_peered":"2023-06-27T12:42:18.516764+0000","last_clean":"2023-06-27T12:42:18.516764+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:18.516764+0000","last_undegraded":"2023-06-27T12:42:18.516764+0000","last_fullsized":"2023-06-27T12:42:18.516764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"242'132","reported_seq":708,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.447002+0000","last_change":"2023-06-27T12:42:12.514010+0000","last_active":"2023-06-27T12:42:16.447002+0000","last_peered":"2023-06-27T12:42:16.447002+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:16.447002+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260756,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":62,"num_read_kb":56880,"num_write":88,"num_write_kb":24408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":380,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-06-27T12:42:18.541748+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:18.541748+0000","last_peered":"2023-06-27T12:42:18.541748+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:18.541748+0000","last_undegraded":"2023-06-27T12:42:18.541748+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"241'322","reported_seq":335,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.448048+0000","last_change":"2023-06-27T12:42:12.515191+0000","last_active":"2023-06-27T12:42:16.448048+0000","last_peered":"2023-06-27T12:42:16.448048+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:16.448048+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3865676,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":17195,"num_write":51,"num_write_kb":15714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":377,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.525853+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:18.525853+0000","last_peered":"2023-06-27T12:42:18.525853+0000","last_clean":"2023-06-27T12:42:18.525853+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:18.525853+0000","last_undegraded":"2023-06-27T12:42:18.525853+0000","last_fullsized":"2023-06-27T12:42:18.525853+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":135,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-06-27T12:42:18.522836+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:18.522836+0000","last_peered":"2023-06-27T12:42:18.522836+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:18.522836+0000","last_undegraded":"2023-06-27T12:42:18.522836+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"238'283","reported_seq":892,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.544690+0000","last_change":"2023-06-27T12:42:18.544690+0000","last_active":"2023-06-27T12:42:18.544690+0000","last_peered":"2023-06-27T12:42:18.544690+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:18.544690+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874228,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":254,"num_read_kb":157621,"num_write":207,"num_write_kb":60840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":383,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.511431+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:18.511431+0000","last_peered":"2023-06-27T12:42:18.511431+0000","last_clean":"2023-06-27T12:42:18.511431+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:18.511431+0000","last_undegraded":"2023-06-27T12:42:18.511431+0000","last_fullsized":"2023-06-27T12:42:18.511431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"220'143","reported_seq":298,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.549869+0000","last_change":"2023-06-27T12:42:18.553600+0000","last_active":"2023-06-27T12:42:18.549869+0000","last_peered":"2023-06-27T12:42:18.549869+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:18.549869+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117781,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":67,"num_read_kb":41522,"num_write":33,"num_write_kb":9435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-06-27T12:42:16.448552+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:16.448552+0000","last_peered":"2023-06-27T12:42:16.448552+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:16.448552+0000","last_undegraded":"2023-06-27T12:42:16.448552+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"230'135","reported_seq":733,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.463610+0000","last_change":"2023-06-27T12:42:12.506312+0000","last_active":"2023-06-27T12:42:16.463610+0000","last_peered":"2023-06-27T12:42:16.463610+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:16.463610+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9193594,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":84,"num_read_kb":96517,"num_write":96,"num_write_kb":28435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"218'93","reported_seq":290,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.542947+0000","last_change":"2023-06-27T12:42:18.545250+0000","last_active":"2023-06-27T12:42:18.542947+0000","last_peered":"2023-06-27T12:42:18.542947+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:18.542947+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2739607,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":50,"num_read_kb":26195,"num_write":26,"num_write_kb":7438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":398,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-06-27T12:42:16.449553+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:16.449553+0000","last_peered":"2023-06-27T12:42:16.449553+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:16.449553+0000","last_undegraded":"2023-06-27T12:42:16.449553+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":377,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.538808+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:18.538808+0000","last_peered":"2023-06-27T12:42:18.538808+0000","last_clean":"2023-06-27T12:42:18.538808+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:18.538808+0000","last_undegraded":"2023-06-27T12:42:18.538808+0000","last_fullsized":"2023-06-27T12:42:18.538808+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"238'168","reported_seq":724,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.447503+0000","last_change":"2023-06-27T12:42:12.552329+0000","last_active":"2023-06-27T12:42:16.447503+0000","last_peered":"2023-06-27T12:42:16.447503+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:16.447503+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6941423,"num_objects":18,"num_object_clones":15,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":79,"num_read_kb":73814,"num_write":84,"num_write_kb":25060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":86137492,"num_objects":237,"num_object_clones":193,"num_object_copies":711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1424,"num_read_kb":988636,"num_write":1237,"num_write_kb":362236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2302,"ondisk_log_size":2302,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":187176,"kb_used_data":156256,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94184664,"statfs":{"total":96636764160,"available":96445095936,"internally_reserved":0,"allocated":160006144,"data_stored":181218304,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":195236,"kb_used_data":164316,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94176604,"statfs":{"total":96636764160,"available":96436842496,"internally_reserved":0,"allocated":168259584,"data_stored":189669412,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":206888,"kb_used_data":175968,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94164952,"statfs":{"total":96636764160,"available":96424910848,"internally_reserved":0,"allocated":180191232,"data_stored":203190180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:21.252 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-06-27T12:42:21.252 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:42:21.253 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:21.372+0000 7fbc06620700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:21.382 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:21.379+0000 7fbc06620700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:21.463 INFO:tasks.rados.rados.0.smithi138.stdout:980: finishing write tid 1 to smithi138138488-37 2023-06-27T12:42:21.463 INFO:tasks.rados.rados.0.smithi138.stdout:980: finishing write tid 2 to smithi138138488-37 2023-06-27T12:42:21.464 INFO:tasks.rados.rados.0.smithi138.stdout:981: done (9 left) 2023-06-27T12:42:21.464 INFO:tasks.rados.rados.0.smithi138.stdout:982: done (8 left) 2023-06-27T12:42:21.464 INFO:tasks.rados.rados.0.smithi138.stdout:983: done (7 left) 2023-06-27T12:42:21.464 INFO:tasks.rados.rados.0.smithi138.stdout:985: done (6 left) 2023-06-27T12:42:21.464 INFO:tasks.rados.rados.0.smithi138.stdout:987: done (5 left) 2023-06-27T12:42:21.464 INFO:tasks.rados.rados.0.smithi138.stdout:989: done (4 left) 2023-06-27T12:42:21.465 INFO:tasks.rados.rados.0.smithi138.stdout:990: write oid 9 current snap is 112 2023-06-27T12:42:21.465 INFO:tasks.rados.rados.0.smithi138.stdout:980: finishing write tid 3 to smithi138138488-37 2023-06-27T12:42:21.465 INFO:tasks.rados.rados.0.smithi138.stdout:980: finishing write tid 4 to smithi138138488-37 2023-06-27T12:42:21.465 INFO:tasks.rados.rados.0.smithi138.stdout:980: oid 37 updating version 0 to 177 2023-06-27T12:42:21.465 INFO:tasks.rados.rados.0.smithi138.stdout:980: oid 37 version 177 is already newer than 175 2023-06-27T12:42:21.465 INFO:tasks.rados.rados.0.smithi138.stdout:980: oid 37 version 177 is already newer than 177 2023-06-27T12:42:21.465 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 177 (ObjNum 477 snap 112 seq_num 477) dirty exists 2023-06-27T12:42:21.465 INFO:tasks.rados.rados.0.smithi138.stdout:980: left oid 37 (ObjNum 477 snap 112 seq_num 477) 2023-06-27T12:42:21.466 INFO:tasks.rados.rados.0.smithi138.stdout:986: finishing write tid 1 to smithi138138488-20 2023-06-27T12:42:21.466 INFO:tasks.rados.rados.0.smithi138.stdout:988: finishing write tid 1 to smithi138138488-45 2023-06-27T12:42:21.466 INFO:tasks.rados.rados.0.smithi138.stdout:990: seq_num 482 ranges {500974=400257,1388909=557169} 2023-06-27T12:42:21.468 INFO:tasks.rados.rados.0.smithi138.stdout:990: writing smithi138138488-9 from 500974 to 901231 tid 1 2023-06-27T12:42:21.475 INFO:tasks.rados.rados.0.smithi138.stdout:990: writing smithi138138488-9 from 1388909 to 1946078 tid 2 2023-06-27T12:42:21.475 INFO:tasks.rados.rados.0.smithi138.stdout:980: done (4 left) 2023-06-27T12:42:21.475 INFO:tasks.rados.rados.0.smithi138.stdout:991: delete oid 23 current snap is 112 2023-06-27T12:42:21.475 INFO:tasks.rados.rados.0.smithi138.stdout:988: finishing write tid 2 to smithi138138488-45 2023-06-27T12:42:21.475 INFO:tasks.rados.rados.0.smithi138.stdout:988: finishing write tid 3 to smithi138138488-45 2023-06-27T12:42:21.475 INFO:tasks.rados.rados.0.smithi138.stdout:988: finishing write tid 4 to smithi138138488-45 2023-06-27T12:42:21.475 INFO:tasks.rados.rados.0.smithi138.stdout:988: oid 45 updating version 0 to 325 2023-06-27T12:42:21.476 INFO:tasks.rados.rados.0.smithi138.stdout:988: oid 45 version 325 is already newer than 325 2023-06-27T12:42:21.476 INFO:tasks.rados.rados.0.smithi138.stdout:988: oid 45 version 325 is already newer than 325 2023-06-27T12:42:21.476 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 325 (ObjNum 480 snap 112 seq_num 480) dirty exists 2023-06-27T12:42:21.476 INFO:tasks.rados.rados.0.smithi138.stdout:988: left oid 45 (ObjNum 480 snap 112 seq_num 480) 2023-06-27T12:42:21.476 INFO:tasks.rados.rados.0.smithi138.stdout:986: finishing write tid 2 to smithi138138488-20 2023-06-27T12:42:21.476 INFO:tasks.rados.rados.0.smithi138.stdout:986: finishing write tid 3 to smithi138138488-20 2023-06-27T12:42:21.476 INFO:tasks.rados.rados.0.smithi138.stdout:984: finishing write tid 1 to smithi138138488-34 2023-06-27T12:42:21.477 INFO:tasks.rados.rados.0.smithi138.stdout:986: finishing write tid 4 to smithi138138488-20 2023-06-27T12:42:21.477 INFO:tasks.rados.rados.0.smithi138.stdout:986: oid 20 updating version 0 to 154 2023-06-27T12:42:21.477 INFO:tasks.rados.rados.0.smithi138.stdout:986: oid 20 updating version 154 to 156 2023-06-27T12:42:21.477 INFO:tasks.rados.rados.0.smithi138.stdout:986: oid 20 version 156 is already newer than 156 2023-06-27T12:42:21.477 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 156 (ObjNum 479 snap 112 seq_num 479) dirty exists 2023-06-27T12:42:21.477 INFO:tasks.rados.rados.0.smithi138.stdout:986: left oid 20 (ObjNum 479 snap 112 seq_num 479) 2023-06-27T12:42:21.477 INFO:tasks.rados.rados.0.smithi138.stdout:984: finishing write tid 2 to smithi138138488-34 2023-06-27T12:42:21.478 INFO:tasks.rados.rados.0.smithi138.stdout:984: finishing write tid 3 to smithi138138488-34 2023-06-27T12:42:21.478 INFO:tasks.rados.rados.0.smithi138.stdout:984: finishing write tid 4 to smithi138138488-34 2023-06-27T12:42:21.478 INFO:tasks.rados.rados.0.smithi138.stdout:984: oid 34 updating version 0 to 305 2023-06-27T12:42:21.478 INFO:tasks.rados.rados.0.smithi138.stdout:984: oid 34 updating version 305 to 307 2023-06-27T12:42:21.478 INFO:tasks.rados.rados.0.smithi138.stdout:984: oid 34 version 307 is already newer than 307 2023-06-27T12:42:21.478 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 307 (ObjNum 478 snap 112 seq_num 478) dirty exists 2023-06-27T12:42:21.478 INFO:tasks.rados.rados.0.smithi138.stdout:984: left oid 34 (ObjNum 478 snap 112 seq_num 478) 2023-06-27T12:42:21.480 INFO:tasks.rados.rados.0.smithi138.stdout:984: done (4 left) 2023-06-27T12:42:21.480 INFO:tasks.rados.rados.0.smithi138.stdout:986: done (3 left) 2023-06-27T12:42:21.480 INFO:tasks.rados.rados.0.smithi138.stdout:988: done (2 left) 2023-06-27T12:42:21.480 INFO:tasks.rados.rados.0.smithi138.stdout:991: done (1 left) 2023-06-27T12:42:21.481 INFO:tasks.rados.rados.0.smithi138.stdout:992: snap_create 2023-06-27T12:42:21.496 INFO:tasks.rados.rados.0.smithi138.stdout:990: finishing write tid 1 to smithi138138488-9 2023-06-27T12:42:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:992: done (1 left) 2023-06-27T12:42:21.509 INFO:tasks.rados.rados.0.smithi138.stdout:993: write (excl) oid 29 current snap is 113 2023-06-27T12:42:21.510 INFO:tasks.rados.rados.0.smithi138.stdout:993: seq_num 484 ranges {534588=550755,1789520=345068} 2023-06-27T12:42:21.515 INFO:tasks.rados.rados.0.smithi138.stdout:993: writing smithi138138488-29 from 534588 to 1085343 tid 1 2023-06-27T12:42:21.522 INFO:tasks.rados.rados.0.smithi138.stdout:993: writing smithi138138488-29 from 1789520 to 2134588 tid 2 2023-06-27T12:42:21.523 INFO:tasks.rados.rados.0.smithi138.stdout:994: read oid 6 snap -1 2023-06-27T12:42:21.523 INFO:tasks.rados.rados.0.smithi138.stdout:994: expect (ObjNum 435 snap 100 seq_num 435) 2023-06-27T12:42:21.523 INFO:tasks.rados.rados.0.smithi138.stdout:995: read oid 12 snap -1 2023-06-27T12:42:21.523 INFO:tasks.rados.rados.0.smithi138.stdout:995: expect (ObjNum 415 snap 98 seq_num 415) 2023-06-27T12:42:21.523 INFO:tasks.rados.rados.0.smithi138.stdout:996: read oid 31 snap -1 2023-06-27T12:42:21.523 INFO:tasks.rados.rados.0.smithi138.stdout:996: expect deleted 2023-06-27T12:42:21.524 INFO:tasks.rados.rados.0.smithi138.stdout:997: read oid 20 snap 112 2023-06-27T12:42:21.524 INFO:tasks.rados.rados.0.smithi138.stdout:997: expect (ObjNum 479 snap 112 seq_num 479) 2023-06-27T12:42:21.524 INFO:tasks.rados.rados.0.smithi138.stdout:998: read oid 2 snap 112 2023-06-27T12:42:21.524 INFO:tasks.rados.rados.0.smithi138.stdout:998: expect deleted 2023-06-27T12:42:21.524 INFO:tasks.rados.rados.0.smithi138.stdout:999: read oid 10 snap -1 2023-06-27T12:42:21.524 INFO:tasks.rados.rados.0.smithi138.stdout:999: expect deleted 2023-06-27T12:42:21.524 INFO:tasks.rados.rados.0.smithi138.stdout:1000: read oid 47 snap -1 2023-06-27T12:42:21.524 INFO:tasks.rados.rados.0.smithi138.stdout:1000: expect (ObjNum 463 snap 108 seq_num 463) 2023-06-27T12:42:21.525 INFO:tasks.rados.rados.0.smithi138.stdout:1001: write (excl) oid 1 current snap is 113 2023-06-27T12:42:21.525 INFO:tasks.rados.rados.0.smithi138.stdout:1001: seq_num 485 ranges {551395=626004,1848002=303393} 2023-06-27T12:42:21.529 INFO:tasks.rados.rados.0.smithi138.stdout:1001: writing smithi138138488-1 from 551395 to 1177399 tid 1 2023-06-27T12:42:21.534 INFO:tasks.rados.rados.0.smithi138.stdout:1001: writing smithi138138488-1 from 1848002 to 2151395 tid 2 2023-06-27T12:42:21.534 INFO:tasks.rados.rados.0.smithi138.stdout:1002: write oid 44 current snap is 113 2023-06-27T12:42:21.535 INFO:tasks.rados.rados.0.smithi138.stdout:1002: seq_num 486 ranges {568202=701253,1906484=261718} 2023-06-27T12:42:21.541 INFO:tasks.rados.rados.0.smithi138.stdout:1002: writing smithi138138488-44 from 568202 to 1269455 tid 1 2023-06-27T12:42:21.547 INFO:tasks.rados.rados.0.smithi138.stdout:1002: writing smithi138138488-44 from 1906484 to 2168202 tid 2 2023-06-27T12:42:21.547 INFO:tasks.rados.rados.0.smithi138.stdout:1003: read oid 11 snap -1 2023-06-27T12:42:21.547 INFO:tasks.rados.rados.0.smithi138.stdout:1003: expect deleted 2023-06-27T12:42:21.547 INFO:tasks.rados.rados.0.smithi138.stdout:1004: read oid 27 snap -1 2023-06-27T12:42:21.548 INFO:tasks.rados.rados.0.smithi138.stdout:1004: expect (ObjNum 452 snap 104 seq_num 452) 2023-06-27T12:42:21.548 INFO:tasks.rados.rados.0.smithi138.stdout:1005: delete oid 38 current snap is 113 2023-06-27T12:42:21.553 INFO:tasks.rados.rados.0.smithi138.stdout:990: finishing write tid 2 to smithi138138488-9 2023-06-27T12:42:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:990: finishing write tid 3 to smithi138138488-9 2023-06-27T12:42:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:990: finishing write tid 4 to smithi138138488-9 2023-06-27T12:42:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:990: oid 9 updating version 0 to 290 2023-06-27T12:42:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:990: oid 9 version 290 is already newer than 290 2023-06-27T12:42:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:990: oid 9 version 290 is already newer than 290 2023-06-27T12:42:21.578 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 290 (ObjNum 482 snap 112 seq_num 482) dirty exists 2023-06-27T12:42:21.579 INFO:tasks.rados.rados.0.smithi138.stdout:990: left oid 9 (ObjNum 482 snap 112 seq_num 482) 2023-06-27T12:42:21.579 INFO:tasks.rados.rados.0.smithi138.stdout:997: expect (ObjNum 479 snap 112 seq_num 479) 2023-06-27T12:42:21.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:21.628 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:21.643 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2023-06-27T12:42:21.509255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":86137492,"num_objects":237,"num_object_clones":193,"num_object_copies":711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1424,"num_read_kb":988636,"num_write":1237,"num_write_kb":362236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2399,"ondisk_log_size":2399,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":770580,"kb_used_data":646900,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376716780,"statfs":{"total":386547056640,"available":385757982720,"internally_reserved":0,"allocated":662425600,"data_stored":751117448,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6000165,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":20,"num_read_kb":13427,"num_write":30,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.081371"},"pg_stats":[{"pgid":"3.f","version":"241'25","reported_seq":583,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.518431+0000","last_change":"2023-06-27T12:42:18.523085+0000","last_active":"2023-06-27T12:42:18.518431+0000","last_peered":"2023-06-27T12:42:18.518431+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:18.518431+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":8,"num_read_kb":4,"num_write":19,"num_write_kb":5014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"241'151","reported_seq":761,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.442699+0000","last_change":"2023-06-27T12:42:12.511488+0000","last_active":"2023-06-27T12:42:16.442699+0000","last_peered":"2023-06-27T12:42:16.442699+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:16.442699+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11311309,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":103,"num_read_kb":107986,"num_write":106,"num_write_kb":31310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"241'174","reported_seq":751,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.548528+0000","last_change":"2023-06-27T12:42:18.553261+0000","last_active":"2023-06-27T12:42:18.548528+0000","last_peered":"2023-06-27T12:42:18.548528+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:18.548528+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9294436,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":127,"num_read_kb":61154,"num_write":121,"num_write_kb":35887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"230'100","reported_seq":716,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.517985+0000","last_change":"2023-06-27T12:42:18.529241+0000","last_active":"2023-06-27T12:42:18.517985+0000","last_peered":"2023-06-27T12:42:18.517985+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:18.517985+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9865907,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":96,"num_read_kb":54526,"num_write":79,"num_write_kb":22429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"231'59","reported_seq":282,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.503221+0000","last_change":"2023-06-27T12:42:18.523206+0000","last_active":"2023-06-27T12:42:18.503221+0000","last_peered":"2023-06-27T12:42:18.503221+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:18.503221+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4722833,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":24677,"num_write":29,"num_write_kb":8919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"241'302","reported_seq":779,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.516330+0000","last_change":"2023-06-27T12:42:18.525961+0000","last_active":"2023-06-27T12:42:18.516330+0000","last_peered":"2023-06-27T12:42:18.516330+0000","last_clean":"2023-06-27T12:42:18.516330+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:18.516330+0000","last_undegraded":"2023-06-27T12:42:18.516330+0000","last_fullsized":"2023-06-27T12:42:18.516330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7748159,"num_objects":34,"num_object_clones":28,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":241,"num_read_kb":149311,"num_write":176,"num_write_kb":51036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"244'153","reported_seq":633,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.532107+0000","last_change":"2023-06-27T12:42:18.532233+0000","last_active":"2023-06-27T12:42:18.532107+0000","last_peered":"2023-06-27T12:42:18.532107+0000","last_clean":"2023-06-27T12:42:18.532107+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:18.532107+0000","last_undegraded":"2023-06-27T12:42:18.532107+0000","last_fullsized":"2023-06-27T12:42:18.532107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2201783,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":109,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"238'62","reported_seq":634,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.524049+0000","last_change":"2023-06-27T12:42:18.527029+0000","last_active":"2023-06-27T12:42:18.524049+0000","last_peered":"2023-06-27T12:42:18.524049+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:18.524049+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":55,"num_read_kb":42879,"num_write":37,"num_write_kb":11692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":398,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-06-27T12:42:16.448959+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:16.448959+0000","last_peered":"2023-06-27T12:42:16.448959+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:16.448959+0000","last_undegraded":"2023-06-27T12:42:16.448959+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":555,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-06-27T12:42:18.548998+0000","last_change":"2023-06-27T12:42:18.552806+0000","last_active":"2023-06-27T12:42:18.548998+0000","last_peered":"2023-06-27T12:42:18.548998+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:18.548998+0000","last_undegraded":"2023-06-27T12:42:18.548998+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":383,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.516764+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:18.516764+0000","last_peered":"2023-06-27T12:42:18.516764+0000","last_clean":"2023-06-27T12:42:18.516764+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:18.516764+0000","last_undegraded":"2023-06-27T12:42:18.516764+0000","last_fullsized":"2023-06-27T12:42:18.516764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"242'132","reported_seq":708,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.447002+0000","last_change":"2023-06-27T12:42:12.514010+0000","last_active":"2023-06-27T12:42:16.447002+0000","last_peered":"2023-06-27T12:42:16.447002+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:16.447002+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260756,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":62,"num_read_kb":56880,"num_write":88,"num_write_kb":24408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":380,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-06-27T12:42:18.541748+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:18.541748+0000","last_peered":"2023-06-27T12:42:18.541748+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:18.541748+0000","last_undegraded":"2023-06-27T12:42:18.541748+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"241'322","reported_seq":335,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.448048+0000","last_change":"2023-06-27T12:42:12.515191+0000","last_active":"2023-06-27T12:42:16.448048+0000","last_peered":"2023-06-27T12:42:16.448048+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:16.448048+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3865676,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":17195,"num_write":51,"num_write_kb":15714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":377,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.525853+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:18.525853+0000","last_peered":"2023-06-27T12:42:18.525853+0000","last_clean":"2023-06-27T12:42:18.525853+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:18.525853+0000","last_undegraded":"2023-06-27T12:42:18.525853+0000","last_fullsized":"2023-06-27T12:42:18.525853+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":135,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-06-27T12:42:18.522836+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:18.522836+0000","last_peered":"2023-06-27T12:42:18.522836+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:18.522836+0000","last_undegraded":"2023-06-27T12:42:18.522836+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"238'283","reported_seq":892,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.544690+0000","last_change":"2023-06-27T12:42:18.544690+0000","last_active":"2023-06-27T12:42:18.544690+0000","last_peered":"2023-06-27T12:42:18.544690+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:18.544690+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874228,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":254,"num_read_kb":157621,"num_write":207,"num_write_kb":60840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":383,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.511431+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:18.511431+0000","last_peered":"2023-06-27T12:42:18.511431+0000","last_clean":"2023-06-27T12:42:18.511431+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:18.511431+0000","last_undegraded":"2023-06-27T12:42:18.511431+0000","last_fullsized":"2023-06-27T12:42:18.511431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"220'143","reported_seq":298,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.549869+0000","last_change":"2023-06-27T12:42:18.553600+0000","last_active":"2023-06-27T12:42:18.549869+0000","last_peered":"2023-06-27T12:42:18.549869+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:18.549869+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117781,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":67,"num_read_kb":41522,"num_write":33,"num_write_kb":9435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-06-27T12:42:16.448552+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:16.448552+0000","last_peered":"2023-06-27T12:42:16.448552+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:16.448552+0000","last_undegraded":"2023-06-27T12:42:16.448552+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"230'135","reported_seq":733,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.463610+0000","last_change":"2023-06-27T12:42:12.506312+0000","last_active":"2023-06-27T12:42:16.463610+0000","last_peered":"2023-06-27T12:42:16.463610+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:16.463610+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9193594,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":84,"num_read_kb":96517,"num_write":96,"num_write_kb":28435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"218'93","reported_seq":290,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:18.542947+0000","last_change":"2023-06-27T12:42:18.545250+0000","last_active":"2023-06-27T12:42:18.542947+0000","last_peered":"2023-06-27T12:42:18.542947+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:18.542947+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2739607,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":50,"num_read_kb":26195,"num_write":26,"num_write_kb":7438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":398,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-06-27T12:42:16.449553+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:16.449553+0000","last_peered":"2023-06-27T12:42:16.449553+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:16.449553+0000","last_undegraded":"2023-06-27T12:42:16.449553+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":377,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-06-27T12:42:18.538808+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:18.538808+0000","last_peered":"2023-06-27T12:42:18.538808+0000","last_clean":"2023-06-27T12:42:18.538808+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:18.538808+0000","last_undegraded":"2023-06-27T12:42:18.538808+0000","last_fullsized":"2023-06-27T12:42:18.538808+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"238'168","reported_seq":724,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:16.447503+0000","last_change":"2023-06-27T12:42:12.552329+0000","last_active":"2023-06-27T12:42:16.447503+0000","last_peered":"2023-06-27T12:42:16.447503+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:16.447503+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6941423,"num_objects":18,"num_object_clones":15,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":79,"num_read_kb":73814,"num_write":84,"num_write_kb":25060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":86137492,"num_objects":237,"num_object_clones":193,"num_object_copies":711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1424,"num_read_kb":988636,"num_write":1237,"num_write_kb":362236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2302,"ondisk_log_size":2302,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":187176,"kb_used_data":156256,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94184664,"statfs":{"total":96636764160,"available":96445095936,"internally_reserved":0,"allocated":160006144,"data_stored":181218304,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":195236,"kb_used_data":164316,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94176604,"statfs":{"total":96636764160,"available":96436842496,"internally_reserved":0,"allocated":168259584,"data_stored":189669412,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":206888,"kb_used_data":175968,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94164952,"statfs":{"total":96636764160,"available":96424910848,"internally_reserved":0,"allocated":180191232,"data_stored":203190180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:21.692 INFO:tasks.rados.rados.0.smithi138.stdout:994: expect (ObjNum 435 snap 100 seq_num 435) 2023-06-27T12:42:21.818 INFO:tasks.rados.rados.0.smithi138.stdout:1000: expect (ObjNum 463 snap 108 seq_num 463) 2023-06-27T12:42:22.005 INFO:tasks.rados.rados.0.smithi138.stdout:995: expect (ObjNum 415 snap 98 seq_num 415) 2023-06-27T12:42:22.116 INFO:tasks.rados.rados.0.smithi138.stdout:993: finishing write tid 1 to smithi138138488-29 2023-06-27T12:42:22.117 INFO:tasks.rados.rados.0.smithi138.stdout:1002: finishing write tid 1 to smithi138138488-44 2023-06-27T12:42:22.117 INFO:tasks.rados.rados.0.smithi138.stdout:1002: finishing write tid 2 to smithi138138488-44 2023-06-27T12:42:22.117 INFO:tasks.rados.rados.0.smithi138.stdout:1004: expect (ObjNum 452 snap 104 seq_num 452) 2023-06-27T12:42:22.252 INFO:tasks.rados.rados.0.smithi138.stdout:1001: finishing write tid 1 to smithi138138488-1 2023-06-27T12:42:22.253 INFO:tasks.rados.rados.0.smithi138.stdout:993: finishing write tid 2 to smithi138138488-29 2023-06-27T12:42:22.253 INFO:tasks.rados.rados.0.smithi138.stdout:993: finishing write tid 3 to smithi138138488-29 2023-06-27T12:42:22.253 INFO:tasks.rados.rados.0.smithi138.stdout:1002: finishing write tid 3 to smithi138138488-44 2023-06-27T12:42:22.253 INFO:tasks.rados.rados.0.smithi138.stdout:993: finishing write tid 4 to smithi138138488-29 2023-06-27T12:42:22.253 INFO:tasks.rados.rados.0.smithi138.stdout:993: oid 29 updating version 0 to 100 2023-06-27T12:42:22.253 INFO:tasks.rados.rados.0.smithi138.stdout:993: oid 29 version 100 is already newer than 100 2023-06-27T12:42:22.253 INFO:tasks.rados.rados.0.smithi138.stdout:993: oid 29 version 100 is already newer than 98 2023-06-27T12:42:22.254 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 100 (ObjNum 484 snap 113 seq_num 484) dirty exists 2023-06-27T12:42:22.254 INFO:tasks.rados.rados.0.smithi138.stdout:993: left oid 29 (ObjNum 484 snap 113 seq_num 484) 2023-06-27T12:42:22.254 INFO:tasks.rados.rados.0.smithi138.stdout:1002: finishing write tid 4 to smithi138138488-44 2023-06-27T12:42:22.254 INFO:tasks.rados.rados.0.smithi138.stdout:1002: oid 44 updating version 0 to 311 2023-06-27T12:42:22.254 INFO:tasks.rados.rados.0.smithi138.stdout:1002: oid 44 version 311 is already newer than 311 2023-06-27T12:42:22.254 INFO:tasks.rados.rados.0.smithi138.stdout:1002: oid 44 version 311 is already newer than 311 2023-06-27T12:42:22.255 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 311 (ObjNum 486 snap 113 seq_num 486) dirty exists 2023-06-27T12:42:22.255 INFO:tasks.rados.rados.0.smithi138.stdout:1002: left oid 44 (ObjNum 486 snap 113 seq_num 486) 2023-06-27T12:42:22.255 INFO:tasks.rados.rados.0.smithi138.stdout:1001: finishing write tid 2 to smithi138138488-1 2023-06-27T12:42:22.255 INFO:tasks.rados.rados.0.smithi138.stdout:1001: finishing write tid 3 to smithi138138488-1 2023-06-27T12:42:22.255 INFO:tasks.rados.rados.0.smithi138.stdout:1001: finishing write tid 4 to smithi138138488-1 2023-06-27T12:42:22.255 INFO:tasks.rados.rados.0.smithi138.stdout:1001: oid 1 updating version 0 to 294 2023-06-27T12:42:22.255 INFO:tasks.rados.rados.0.smithi138.stdout:1001: oid 1 version 294 is already newer than 291 2023-06-27T12:42:22.255 INFO:tasks.rados.rados.0.smithi138.stdout:1001: oid 1 version 294 is already newer than 294 2023-06-27T12:42:22.256 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 294 (ObjNum 485 snap 113 seq_num 485) dirty exists 2023-06-27T12:42:22.256 INFO:tasks.rados.rados.0.smithi138.stdout:1001: left oid 1 (ObjNum 485 snap 113 seq_num 485) 2023-06-27T12:42:22.256 INFO:tasks.rados.rados.0.smithi138.stdout:990: done (13 left) 2023-06-27T12:42:22.256 INFO:tasks.rados.rados.0.smithi138.stdout:993: done (12 left) 2023-06-27T12:42:22.256 INFO:tasks.rados.rados.0.smithi138.stdout:994: done (11 left) 2023-06-27T12:42:22.256 INFO:tasks.rados.rados.0.smithi138.stdout:995: done (10 left) 2023-06-27T12:42:22.256 INFO:tasks.rados.rados.0.smithi138.stdout:996: done (9 left) 2023-06-27T12:42:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:997: done (8 left) 2023-06-27T12:42:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:998: done (7 left) 2023-06-27T12:42:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:999: done (6 left) 2023-06-27T12:42:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:1000: done (5 left) 2023-06-27T12:42:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:1001: done (4 left) 2023-06-27T12:42:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:1002: done (3 left) 2023-06-27T12:42:22.257 INFO:tasks.rados.rados.0.smithi138.stdout:1003: done (2 left) 2023-06-27T12:42:22.258 INFO:tasks.rados.rados.0.smithi138.stdout:1004: done (1 left) 2023-06-27T12:42:22.258 INFO:tasks.rados.rados.0.smithi138.stdout:1005: done (0 left) 2023-06-27T12:42:22.258 INFO:tasks.rados.rados.0.smithi138.stdout:1006: write (excl) oid 11 current snap is 113 2023-06-27T12:42:22.258 INFO:tasks.rados.rados.0.smithi138.stdout:1006: seq_num 488 ranges {601816=568104,1623448=451823} 2023-06-27T12:42:22.258 INFO:tasks.rados.rados.0.smithi138.stdout:1006: writing smithi138138488-11 from 601816 to 1169920 tid 1 2023-06-27T12:42:22.263 INFO:tasks.rados.rados.0.smithi138.stdout:1006: writing smithi138138488-11 from 1623448 to 2075271 tid 2 2023-06-27T12:42:22.263 INFO:tasks.rados.rados.0.smithi138.stdout:1007: read oid 25 snap 112 2023-06-27T12:42:22.263 INFO:tasks.rados.rados.0.smithi138.stdout:1007: expect (ObjNum 401 snap 98 seq_num 401) 2023-06-27T12:42:22.263 INFO:tasks.rados.rados.0.smithi138.stdout:1008: delete oid 15 current snap is 113 2023-06-27T12:42:22.287 INFO:tasks.rados.rados.0.smithi138.stdout:1006: finishing write tid 1 to smithi138138488-11 2023-06-27T12:42:22.307 INFO:tasks.rados.rados.0.smithi138.stdout:1006: finishing write tid 2 to smithi138138488-11 2023-06-27T12:42:22.312 INFO:tasks.rados.rados.0.smithi138.stdout:1007: expect (ObjNum 401 snap 98 seq_num 401) 2023-06-27T12:42:22.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:22.458 INFO:tasks.rados.rados.0.smithi138.stdout:1007: done (2 left) 2023-06-27T12:42:22.458 INFO:tasks.rados.rados.0.smithi138.stdout:1008: done (1 left) 2023-06-27T12:42:22.458 INFO:tasks.rados.rados.0.smithi138.stdout:1009: snap_remove snap 112 2023-06-27T12:42:22.516 INFO:tasks.rados.rados.0.smithi138.stdout:1009: done (1 left) 2023-06-27T12:42:22.516 INFO:tasks.rados.rados.0.smithi138.stdout:1010: write oid 30 current snap is 113 2023-06-27T12:42:22.517 INFO:tasks.rados.rados.0.smithi138.stdout:1006: finishing write tid 3 to smithi138138488-11 2023-06-27T12:42:22.517 INFO:tasks.rados.rados.0.smithi138.stdout:1010: seq_num 490 ranges {635430=718602,2140412=95018} 2023-06-27T12:42:22.523 INFO:tasks.rados.rados.0.smithi138.stdout:1010: writing smithi138138488-30 from 635430 to 1354032 tid 1 2023-06-27T12:42:22.529 INFO:tasks.rados.rados.0.smithi138.stdout:1010: writing smithi138138488-30 from 2140412 to 2235430 tid 2 2023-06-27T12:42:22.529 INFO:tasks.rados.rados.0.smithi138.stdout:1011: write (excl) oid 12 current snap is 113 2023-06-27T12:42:22.529 INFO:tasks.rados.rados.0.smithi138.stdout:1006: finishing write tid 4 to smithi138138488-11 2023-06-27T12:42:22.529 INFO:tasks.rados.rados.0.smithi138.stdout:1006: oid 11 updating version 0 to 180 2023-06-27T12:42:22.529 INFO:tasks.rados.rados.0.smithi138.stdout:1006: oid 11 version 180 is already newer than 178 2023-06-27T12:42:22.529 INFO:tasks.rados.rados.0.smithi138.stdout:1006: oid 11 version 180 is already newer than 180 2023-06-27T12:42:22.530 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 11 v 180 (ObjNum 488 snap 113 seq_num 488) dirty exists 2023-06-27T12:42:22.530 INFO:tasks.rados.rados.0.smithi138.stdout:1006: left oid 11 (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:42:22.530 INFO:tasks.rados.rados.0.smithi138.stdout:1011: seq_num 491 ranges {652237=793851,2198894=53343} 2023-06-27T12:42:22.540 INFO:tasks.rados.rados.0.smithi138.stdout:1011: writing smithi138138488-12 from 652237 to 1446088 tid 1 2023-06-27T12:42:22.545 INFO:tasks.rados.rados.0.smithi138.stdout:1011: writing smithi138138488-12 from 2198894 to 2252237 tid 2 2023-06-27T12:42:22.545 INFO:tasks.rados.rados.0.smithi138.stdout:1006: done (2 left) 2023-06-27T12:42:22.545 INFO:tasks.rados.rados.0.smithi138.stdout:1012: read oid 19 snap -1 2023-06-27T12:42:22.545 INFO:tasks.rados.rados.0.smithi138.stdout:1012: expect deleted 2023-06-27T12:42:22.545 INFO:tasks.rados.rados.0.smithi138.stdout:1013: write oid 22 current snap is 113 2023-06-27T12:42:22.546 INFO:tasks.rados.rados.0.smithi138.stdout:1013: seq_num 492 ranges {669044=469100,1741023=459161} 2023-06-27T12:42:22.553 INFO:tasks.rados.rados.0.smithi138.stdout:1013: writing smithi138138488-22 from 669044 to 1138144 tid 1 2023-06-27T12:42:22.559 INFO:tasks.rados.rados.0.smithi138.stdout:1013: writing smithi138138488-22 from 1741023 to 2200184 tid 2 2023-06-27T12:42:22.560 INFO:tasks.rados.rados.0.smithi138.stdout:1014: write oid 50 current snap is 113 2023-06-27T12:42:22.560 INFO:tasks.rados.rados.0.smithi138.stdout:1014: seq_num 493 ranges {685851=544349,1799505=486346} 2023-06-27T12:42:22.566 INFO:tasks.rados.rados.0.smithi138.stdout:1014: writing smithi138138488-50 from 685851 to 1230200 tid 1 2023-06-27T12:42:22.572 INFO:tasks.rados.rados.0.smithi138.stdout:1014: writing smithi138138488-50 from 1799505 to 2285851 tid 2 2023-06-27T12:42:22.573 INFO:tasks.rados.rados.0.smithi138.stdout:1015: delete oid 7 current snap is 113 2023-06-27T12:42:22.573 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:22.568+0000 7f090d147700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:22.576 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:22.573+0000 7f090d147700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:22.613 INFO:tasks.rados.rados.0.smithi138.stdout:1010: finishing write tid 1 to smithi138138488-30 2023-06-27T12:42:22.626 INFO:tasks.rados.rados.0.smithi138.stdout:1013: finishing write tid 1 to smithi138138488-22 2023-06-27T12:42:22.640 INFO:tasks.rados.rados.0.smithi138.stdout:1010: finishing write tid 2 to smithi138138488-30 2023-06-27T12:42:22.667 INFO:tasks.rados.rados.0.smithi138.stdout:1013: finishing write tid 2 to smithi138138488-22 2023-06-27T12:42:22.667 INFO:tasks.rados.rados.0.smithi138.stdout:1013: finishing write tid 3 to smithi138138488-22 2023-06-27T12:42:22.667 INFO:tasks.rados.rados.0.smithi138.stdout:1013: finishing write tid 4 to smithi138138488-22 2023-06-27T12:42:22.667 INFO:tasks.rados.rados.0.smithi138.stdout:1013: oid 22 updating version 0 to 173 2023-06-27T12:42:22.667 INFO:tasks.rados.rados.0.smithi138.stdout:1013: oid 22 version 173 is already newer than 173 2023-06-27T12:42:22.668 INFO:tasks.rados.rados.0.smithi138.stdout:1013: oid 22 version 173 is already newer than 173 2023-06-27T12:42:22.668 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 173 (ObjNum 492 snap 113 seq_num 492) dirty exists 2023-06-27T12:42:22.668 INFO:tasks.rados.rados.0.smithi138.stdout:1013: left oid 22 (ObjNum 492 snap 113 seq_num 492) 2023-06-27T12:42:22.681 INFO:tasks.rados.rados.0.smithi138.stdout:1010: finishing write tid 3 to smithi138138488-30 2023-06-27T12:42:22.688 INFO:tasks.rados.rados.0.smithi138.stdout:1010: finishing write tid 4 to smithi138138488-30 2023-06-27T12:42:22.688 INFO:tasks.rados.rados.0.smithi138.stdout:1010: oid 30 updating version 0 to 148 2023-06-27T12:42:22.688 INFO:tasks.rados.rados.0.smithi138.stdout:1010: oid 30 version 148 is already newer than 148 2023-06-27T12:42:22.689 INFO:tasks.rados.rados.0.smithi138.stdout:1010: oid 30 version 148 is already newer than 148 2023-06-27T12:42:22.689 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 148 (ObjNum 490 snap 113 seq_num 490) dirty exists 2023-06-27T12:42:22.689 INFO:tasks.rados.rados.0.smithi138.stdout:1010: left oid 30 (ObjNum 490 snap 113 seq_num 490) 2023-06-27T12:42:22.689 INFO:tasks.rados.rados.0.smithi138.stdout:1011: finishing write tid 1 to smithi138138488-12 2023-06-27T12:42:22.691 INFO:tasks.rados.rados.0.smithi138.stdout:1014: finishing write tid 1 to smithi138138488-50 2023-06-27T12:42:22.694 INFO:tasks.rados.rados.0.smithi138.stdout:1010: done (5 left) 2023-06-27T12:42:22.694 INFO:tasks.rados.rados.0.smithi138.stdout:1012: done (4 left) 2023-06-27T12:42:22.694 INFO:tasks.rados.rados.0.smithi138.stdout:1013: done (3 left) 2023-06-27T12:42:22.694 INFO:tasks.rados.rados.0.smithi138.stdout:1015: done (2 left) 2023-06-27T12:42:22.695 INFO:tasks.rados.rados.0.smithi138.stdout:1016: delete oid 43 current snap is 113 2023-06-27T12:42:22.697 INFO:tasks.rados.rados.0.smithi138.stdout:1016: done (2 left) 2023-06-27T12:42:22.697 INFO:tasks.rados.rados.0.smithi138.stdout:1017: delete oid 25 current snap is 113 2023-06-27T12:42:22.708 INFO:tasks.rados.rados.0.smithi138.stdout:1014: finishing write tid 2 to smithi138138488-50 2023-06-27T12:42:22.748 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:22.749 INFO:tasks.rados.rados.0.smithi138.stdout:1014: finishing write tid 3 to smithi138138488-50 2023-06-27T12:42:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:1011: finishing write tid 2 to smithi138138488-12 2023-06-27T12:42:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:1014: finishing write tid 4 to smithi138138488-50 2023-06-27T12:42:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:1014: oid 50 updating version 0 to 67 2023-06-27T12:42:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:1014: oid 50 version 67 is already newer than 67 2023-06-27T12:42:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:1014: oid 50 version 67 is already newer than 67 2023-06-27T12:42:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 67 (ObjNum 493 snap 113 seq_num 493) dirty exists 2023-06-27T12:42:22.750 INFO:tasks.rados.rados.0.smithi138.stdout:1014: left oid 50 (ObjNum 493 snap 113 seq_num 493) 2023-06-27T12:42:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:1014: done (2 left) 2023-06-27T12:42:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:1017: done (1 left) 2023-06-27T12:42:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:1018: delete oid 41 current snap is 113 2023-06-27T12:42:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:1011: finishing write tid 3 to smithi138138488-12 2023-06-27T12:42:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:1011: finishing write tid 4 to smithi138138488-12 2023-06-27T12:42:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:1011: oid 12 updating version 0 to 183 2023-06-27T12:42:22.751 INFO:tasks.rados.rados.0.smithi138.stdout:1011: oid 12 version 183 is already newer than 181 2023-06-27T12:42:22.752 INFO:tasks.rados.rados.0.smithi138.stdout:1011: oid 12 version 183 is already newer than 183 2023-06-27T12:42:22.752 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 183 (ObjNum 491 snap 113 seq_num 491) dirty exists 2023-06-27T12:42:22.752 INFO:tasks.rados.rados.0.smithi138.stdout:1011: left oid 12 (ObjNum 491 snap 113 seq_num 491) 2023-06-27T12:42:22.752 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:22.753 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:22.759 INFO:tasks.rados.rados.0.smithi138.stdout:1011: done (1 left) 2023-06-27T12:42:22.759 INFO:tasks.rados.rados.0.smithi138.stdout:1018: done (0 left) 2023-06-27T12:42:22.759 INFO:tasks.rados.rados.0.smithi138.stdout:1019: delete oid 44 current snap is 113 2023-06-27T12:42:22.808 INFO:tasks.rados.rados.0.smithi138.stdout:1019: done (0 left) 2023-06-27T12:42:22.808 INFO:tasks.rados.rados.0.smithi138.stdout:1020: write (excl) oid 14 current snap is 113 2023-06-27T12:42:22.808 INFO:tasks.rados.rados.0.smithi138.stdout:1020: seq_num 499 ranges {786693=712196,2150397=236296} 2023-06-27T12:42:22.816 INFO:tasks.rados.rados.0.smithi138.stdout:1020: writing smithi138138488-14 from 786693 to 1498889 tid 1 2023-06-27T12:42:22.821 INFO:tasks.rados.rados.0.smithi138.stdout:1020: writing smithi138138488-14 from 2150397 to 2386693 tid 2 2023-06-27T12:42:22.821 INFO:tasks.rados.rados.0.smithi138.stdout:1021: snap_create 2023-06-27T12:42:22.864 INFO:tasks.rados.rados.0.smithi138.stdout:1020: finishing write tid 1 to smithi138138488-14 2023-06-27T12:42:22.912 INFO:tasks.rados.rados.0.smithi138.stdout:1020: finishing write tid 2 to smithi138138488-14 2023-06-27T12:42:22.912 INFO:tasks.rados.rados.0.smithi138.stdout:1020: finishing write tid 3 to smithi138138488-14 2023-06-27T12:42:22.913 INFO:tasks.rados.rados.0.smithi138.stdout:1020: finishing write tid 4 to smithi138138488-14 2023-06-27T12:42:22.913 INFO:tasks.rados.rados.0.smithi138.stdout:1020: oid 14 updating version 0 to 299 2023-06-27T12:42:22.914 INFO:tasks.rados.rados.0.smithi138.stdout:1020: oid 14 version 299 is already newer than 297 2023-06-27T12:42:22.914 INFO:tasks.rados.rados.0.smithi138.stdout:1020: oid 14 version 299 is already newer than 299 2023-06-27T12:42:22.914 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 299 (ObjNum 499 snap 113 seq_num 499) dirty exists 2023-06-27T12:42:22.914 INFO:tasks.rados.rados.0.smithi138.stdout:1020: left oid 14 (ObjNum 499 snap 113 seq_num 499) 2023-06-27T12:42:23.520 INFO:tasks.rados.rados.0.smithi138.stdout:1020: done (1 left) 2023-06-27T12:42:23.520 INFO:tasks.rados.rados.0.smithi138.stdout:1021: done (0 left) 2023-06-27T12:42:23.520 INFO:tasks.rados.rados.0.smithi138.stdout:1022: delete oid 1 current snap is 114 2023-06-27T12:42:23.599 INFO:tasks.rados.rados.0.smithi138.stdout:1022: done (0 left) 2023-06-27T12:42:23.599 INFO:tasks.rados.rados.0.smithi138.stdout:1023: write oid 25 current snap is 114 2023-06-27T12:42:23.599 INFO:tasks.rados.rados.0.smithi138.stdout:1023: seq_num 501 ranges {420307=462694,1351008=617495} 2023-06-27T12:42:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:1023: writing smithi138138488-25 from 420307 to 883001 tid 1 2023-06-27T12:42:23.610 INFO:tasks.rados.rados.0.smithi138.stdout:1023: writing smithi138138488-25 from 1351008 to 1968503 tid 2 2023-06-27T12:42:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:1024: write oid 37 current snap is 114 2023-06-27T12:42:23.611 INFO:tasks.rados.rados.0.smithi138.stdout:1024: seq_num 502 ranges {437114=654296,1525843=477506} 2023-06-27T12:42:23.616 INFO:tasks.rados.rados.0.smithi138.stdout:1024: writing smithi138138488-37 from 437114 to 1091410 tid 1 2023-06-27T12:42:23.623 INFO:tasks.rados.rados.0.smithi138.stdout:1024: writing smithi138138488-37 from 1525843 to 2003349 tid 2 2023-06-27T12:42:23.623 INFO:tasks.rados.rados.0.smithi138.stdout:1025: snap_create 2023-06-27T12:42:23.642 INFO:tasks.rados.rados.0.smithi138.stdout:1023: finishing write tid 1 to smithi138138488-25 2023-06-27T12:42:23.683 INFO:tasks.rados.rados.0.smithi138.stdout:1023: finishing write tid 2 to smithi138138488-25 2023-06-27T12:42:23.683 INFO:tasks.rados.rados.0.smithi138.stdout:1023: finishing write tid 3 to smithi138138488-25 2023-06-27T12:42:23.683 INFO:tasks.rados.rados.0.smithi138.stdout:1023: finishing write tid 4 to smithi138138488-25 2023-06-27T12:42:23.683 INFO:tasks.rados.rados.0.smithi138.stdout:1023: oid 25 updating version 0 to 104 2023-06-27T12:42:23.684 INFO:tasks.rados.rados.0.smithi138.stdout:1023: oid 25 version 104 is already newer than 104 2023-06-27T12:42:23.684 INFO:tasks.rados.rados.0.smithi138.stdout:1023: oid 25 version 104 is already newer than 104 2023-06-27T12:42:23.684 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 104 (ObjNum 501 snap 114 seq_num 501) dirty exists 2023-06-27T12:42:23.684 INFO:tasks.rados.rados.0.smithi138.stdout:1023: left oid 25 (ObjNum 501 snap 114 seq_num 501) 2023-06-27T12:42:23.684 INFO:tasks.rados.rados.0.smithi138.stdout:1024: finishing write tid 1 to smithi138138488-37 2023-06-27T12:42:23.684 INFO:tasks.rados.rados.0.smithi138.stdout:1024: finishing write tid 2 to smithi138138488-37 2023-06-27T12:42:23.684 INFO:tasks.rados.rados.0.smithi138.stdout:1024: finishing write tid 3 to smithi138138488-37 2023-06-27T12:42:23.685 INFO:tasks.rados.rados.0.smithi138.stdout:1024: finishing write tid 4 to smithi138138488-37 2023-06-27T12:42:23.685 INFO:tasks.rados.rados.0.smithi138.stdout:1024: oid 37 updating version 0 to 187 2023-06-27T12:42:23.685 INFO:tasks.rados.rados.0.smithi138.stdout:1024: oid 37 version 187 is already newer than 187 2023-06-27T12:42:23.685 INFO:tasks.rados.rados.0.smithi138.stdout:1024: oid 37 version 187 is already newer than 187 2023-06-27T12:42:23.685 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 187 (ObjNum 502 snap 114 seq_num 502) dirty exists 2023-06-27T12:42:23.685 INFO:tasks.rados.rados.0.smithi138.stdout:1024: left oid 37 (ObjNum 502 snap 114 seq_num 502) 2023-06-27T12:42:24.525 INFO:tasks.rados.rados.0.smithi138.stdout:1023: done (2 left) 2023-06-27T12:42:24.526 INFO:tasks.rados.rados.0.smithi138.stdout:1024: done (1 left) 2023-06-27T12:42:24.526 INFO:tasks.rados.rados.0.smithi138.stdout:1025: done (0 left) 2023-06-27T12:42:24.526 INFO:tasks.rados.rados.0.smithi138.stdout:1026: write oid 23 current snap is 115 2023-06-27T12:42:24.526 INFO:tasks.rados.rados.0.smithi138.stdout:1026: seq_num 503 ranges {453921=729545,1584325=621164} 2023-06-27T12:42:24.531 INFO:tasks.rados.rados.0.smithi138.stdout:1026: writing smithi138138488-23 from 453921 to 1183466 tid 1 2023-06-27T12:42:24.537 INFO:tasks.rados.rados.0.smithi138.stdout:1026: writing smithi138138488-23 from 1584325 to 2205489 tid 2 2023-06-27T12:42:24.538 INFO:tasks.rados.rados.0.smithi138.stdout:1027: snap_create 2023-06-27T12:42:24.571 INFO:tasks.rados.rados.0.smithi138.stdout:1026: finishing write tid 1 to smithi138138488-23 2023-06-27T12:42:24.612 INFO:tasks.rados.rados.0.smithi138.stdout:1026: finishing write tid 2 to smithi138138488-23 2023-06-27T12:42:24.616 INFO:tasks.rados.rados.0.smithi138.stdout:1026: finishing write tid 3 to smithi138138488-23 2023-06-27T12:42:24.618 INFO:tasks.rados.rados.0.smithi138.stdout:1026: finishing write tid 4 to smithi138138488-23 2023-06-27T12:42:24.618 INFO:tasks.rados.rados.0.smithi138.stdout:1026: oid 23 updating version 0 to 139 2023-06-27T12:42:24.618 INFO:tasks.rados.rados.0.smithi138.stdout:1026: oid 23 version 139 is already newer than 139 2023-06-27T12:42:24.618 INFO:tasks.rados.rados.0.smithi138.stdout:1026: oid 23 version 139 is already newer than 139 2023-06-27T12:42:24.618 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 139 (ObjNum 503 snap 115 seq_num 503) dirty exists 2023-06-27T12:42:24.619 INFO:tasks.rados.rados.0.smithi138.stdout:1026: left oid 23 (ObjNum 503 snap 115 seq_num 503) 2023-06-27T12:42:24.754 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:24.894 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:24.889+0000 7f4d053b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:24.897 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:24.893+0000 7f4d053b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:24.947 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:24.949 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:24.950 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:25.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~110s 2023-06-27T12:42:25.429 INFO:tasks.rados.rados.0.smithi138.stdout:1026: done (1 left) 2023-06-27T12:42:25.430 INFO:tasks.rados.rados.0.smithi138.stdout:1027: done (0 left) 2023-06-27T12:42:25.430 INFO:tasks.rados.rados.0.smithi138.stdout:1028: read oid 17 snap -1 2023-06-27T12:42:25.430 INFO:tasks.rados.rados.0.smithi138.stdout:1028: expect deleted 2023-06-27T12:42:25.430 INFO:tasks.rados.rados.0.smithi138.stdout:1029: read oid 50 snap -1 2023-06-27T12:42:25.430 INFO:tasks.rados.rados.0.smithi138.stdout:1029: expect (ObjNum 493 snap 113 seq_num 493) 2023-06-27T12:42:25.430 INFO:tasks.rados.rados.0.smithi138.stdout:1030: write oid 48 current snap is 116 2023-06-27T12:42:25.430 INFO:tasks.rados.rados.0.smithi138.stdout:1030: seq_num 504 ranges {470728=404794,1526454=481175} 2023-06-27T12:42:25.434 INFO:tasks.rados.rados.0.smithi138.stdout:1030: writing smithi138138488-48 from 470728 to 875522 tid 1 2023-06-27T12:42:25.441 INFO:tasks.rados.rados.0.smithi138.stdout:1030: writing smithi138138488-48 from 1526454 to 2007629 tid 2 2023-06-27T12:42:25.441 INFO:tasks.rados.rados.0.smithi138.stdout:1031: write oid 30 current snap is 116 2023-06-27T12:42:25.441 INFO:tasks.rados.rados.0.smithi138.stdout:1031: seq_num 505 ranges {487535=480043,1584936=624833} 2023-06-27T12:42:25.446 INFO:tasks.rados.rados.0.smithi138.stdout:1031: writing smithi138138488-30 from 487535 to 967578 tid 1 2023-06-27T12:42:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:1031: writing smithi138138488-30 from 1584936 to 2209769 tid 2 2023-06-27T12:42:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:1032: snap_remove snap 114 2023-06-27T12:42:26.434 INFO:tasks.rados.rados.0.smithi138.stdout:1032: done (4 left) 2023-06-27T12:42:26.435 INFO:tasks.rados.rados.0.smithi138.stdout:1033: read oid 2 snap -1 2023-06-27T12:42:26.435 INFO:tasks.rados.rados.0.smithi138.stdout:1033: expect deleted 2023-06-27T12:42:26.435 INFO:tasks.rados.rados.0.smithi138.stdout:1034: read oid 29 snap -1 2023-06-27T12:42:26.435 INFO:tasks.rados.rados.0.smithi138.stdout:1034: expect (ObjNum 484 snap 113 seq_num 484) 2023-06-27T12:42:26.435 INFO:tasks.rados.rados.0.smithi138.stdout:1035: read oid 32 snap 113 2023-06-27T12:42:26.435 INFO:tasks.rados.rados.0.smithi138.stdout:1035: expect (ObjNum 374 snap 91 seq_num 374) 2023-06-27T12:42:26.435 INFO:tasks.rados.rados.0.smithi138.stdout:1036: delete oid 31 current snap is 116 2023-06-27T12:42:26.436 INFO:tasks.rados.rados.0.smithi138.stdout:1029: expect (ObjNum 493 snap 113 seq_num 493) 2023-06-27T12:42:26.540 INFO:tasks.rados.rados.0.smithi138.stdout:1030: finishing write tid 1 to smithi138138488-48 2023-06-27T12:42:26.540 INFO:tasks.rados.rados.0.smithi138.stdout:1030: finishing write tid 2 to smithi138138488-48 2023-06-27T12:42:26.540 INFO:tasks.rados.rados.0.smithi138.stdout:1030: finishing write tid 3 to smithi138138488-48 2023-06-27T12:42:26.541 INFO:tasks.rados.rados.0.smithi138.stdout:1030: finishing write tid 4 to smithi138138488-48 2023-06-27T12:42:26.541 INFO:tasks.rados.rados.0.smithi138.stdout:1030: oid 48 updating version 0 to 177 2023-06-27T12:42:26.541 INFO:tasks.rados.rados.0.smithi138.stdout:1030: oid 48 version 177 is already newer than 177 2023-06-27T12:42:26.541 INFO:tasks.rados.rados.0.smithi138.stdout:1030: oid 48 version 177 is already newer than 177 2023-06-27T12:42:26.541 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 177 (ObjNum 504 snap 116 seq_num 504) dirty exists 2023-06-27T12:42:26.541 INFO:tasks.rados.rados.0.smithi138.stdout:1030: left oid 48 (ObjNum 504 snap 116 seq_num 504) 2023-06-27T12:42:26.541 INFO:tasks.rados.rados.0.smithi138.stdout:1031: finishing write tid 1 to smithi138138488-30 2023-06-27T12:42:26.542 INFO:tasks.rados.rados.0.smithi138.stdout:1031: finishing write tid 2 to smithi138138488-30 2023-06-27T12:42:26.542 INFO:tasks.rados.rados.0.smithi138.stdout:1031: finishing write tid 3 to smithi138138488-30 2023-06-27T12:42:26.542 INFO:tasks.rados.rados.0.smithi138.stdout:1031: finishing write tid 4 to smithi138138488-30 2023-06-27T12:42:26.542 INFO:tasks.rados.rados.0.smithi138.stdout:1031: oid 30 updating version 0 to 152 2023-06-27T12:42:26.542 INFO:tasks.rados.rados.0.smithi138.stdout:1031: oid 30 version 152 is already newer than 152 2023-06-27T12:42:26.542 INFO:tasks.rados.rados.0.smithi138.stdout:1031: oid 30 version 152 is already newer than 152 2023-06-27T12:42:26.542 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 152 (ObjNum 505 snap 116 seq_num 505) dirty exists 2023-06-27T12:42:26.542 INFO:tasks.rados.rados.0.smithi138.stdout:1031: left oid 30 (ObjNum 505 snap 116 seq_num 505) 2023-06-27T12:42:26.543 INFO:tasks.rados.rados.0.smithi138.stdout:1028: done (7 left) 2023-06-27T12:42:26.543 INFO:tasks.rados.rados.0.smithi138.stdout:1029: done (6 left) 2023-06-27T12:42:26.543 INFO:tasks.rados.rados.0.smithi138.stdout:1030: done (5 left) 2023-06-27T12:42:26.543 INFO:tasks.rados.rados.0.smithi138.stdout:1031: done (4 left) 2023-06-27T12:42:26.543 INFO:tasks.rados.rados.0.smithi138.stdout:1036: done (3 left) 2023-06-27T12:42:26.543 INFO:tasks.rados.rados.0.smithi138.stdout:1037: write oid 16 current snap is 116 2023-06-27T12:42:26.543 INFO:tasks.rados.rados.0.smithi138.stdout:1037: seq_num 507 ranges {521149=630541,1701900=628502} 2023-06-27T12:42:26.546 INFO:tasks.rados.rados.0.smithi138.stdout:1037: writing smithi138138488-16 from 521149 to 1151690 tid 1 2023-06-27T12:42:26.553 INFO:tasks.rados.rados.0.smithi138.stdout:1037: writing smithi138138488-16 from 1701900 to 2330402 tid 2 2023-06-27T12:42:26.553 INFO:tasks.rados.rados.0.smithi138.stdout:1033: done (3 left) 2023-06-27T12:42:26.553 INFO:tasks.rados.rados.0.smithi138.stdout:1038: read oid 35 snap -1 2023-06-27T12:42:26.553 INFO:tasks.rados.rados.0.smithi138.stdout:1038: expect (ObjNum 366 snap 91 seq_num 366) 2023-06-27T12:42:26.553 INFO:tasks.rados.rados.0.smithi138.stdout:1039: snap_remove snap 115 2023-06-27T12:42:26.647 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:42:26.647 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:42:26.648 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:26.775 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:26.768+0000 7fb012673700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:26.778 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:26.774+0000 7fb012673700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:26.951 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:26.976 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:26.988 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2023-06-27T12:42:26.428766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75448768,"num_objects":242,"num_object_clones":198,"num_object_copies":726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1489,"num_read_kb":1049473,"num_write":1301,"num_write_kb":377185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2489,"ondisk_log_size":2489,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":762392,"kb_used_data":629892,"kb_used_omap":0,"kb_used_meta":123648,"kb_avail":376724968,"statfs":{"total":386547056640,"available":385766367232,"internally_reserved":0,"allocated":645009408,"data_stored":722311594,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":126615552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10688724,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":60837,"num_write":64,"num_write_kb":14949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.917276"},"pg_stats":[{"pgid":"3.f","version":"248'29","reported_seq":594,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:23.526927+0000","last_change":"2023-06-27T12:42:22.601231+0000","last_active":"2023-06-27T12:42:23.526927+0000","last_peered":"2023-06-27T12:42:23.526927+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:23.526927+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":8,"num_read_kb":4,"num_write":19,"num_write_kb":5014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"247'156","reported_seq":786,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:22.534556+0000","last_change":"2023-06-27T12:42:22.534556+0000","last_active":"2023-06-27T12:42:22.534556+0000","last_peered":"2023-06-27T12:42:22.534556+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:22.534556+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9428958,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":113,"num_read_kb":130147,"num_write":110,"num_write_kb":32299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"250'187","reported_seq":783,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:23.663948+0000","last_change":"2023-06-27T12:42:22.545108+0000","last_active":"2023-06-27T12:42:23.663948+0000","last_peered":"2023-06-27T12:42:23.663948+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:23.663948+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13210698,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":68222,"num_write":137,"num_write_kb":39629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"230'100","reported_seq":727,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:23.541796+0000","last_change":"2023-06-27T12:42:22.640862+0000","last_active":"2023-06-27T12:42:23.541796+0000","last_peered":"2023-06-27T12:42:23.541796+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:23.541796+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9865907,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":96,"num_read_kb":54526,"num_write":79,"num_write_kb":22429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"248'65","reported_seq":297,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:23.539969+0000","last_change":"2023-06-27T12:42:22.662135+0000","last_active":"2023-06-27T12:42:23.539969+0000","last_peered":"2023-06-27T12:42:23.539969+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:23.539969+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1495823,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":67,"num_read_kb":34135,"num_write":29,"num_write_kb":8919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"249'314","reported_seq":799,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-06-27T12:42:23.539016+0000","last_change":"2023-06-27T12:42:22.637820+0000","last_active":"2023-06-27T12:42:23.539016+0000","last_peered":"2023-06-27T12:42:23.539016+0000","last_clean":"2023-06-27T12:42:23.539016+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:23.539016+0000","last_undegraded":"2023-06-27T12:42:23.539016+0000","last_fullsized":"2023-06-27T12:42:23.539016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5949843,"num_objects":35,"num_object_clones":29,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":244,"num_read_kb":149313,"num_write":184,"num_write_kb":52819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"249'154","reported_seq":647,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-06-27T12:42:23.530917+0000","last_change":"2023-06-27T12:42:22.636608+0000","last_active":"2023-06-27T12:42:23.530917+0000","last_peered":"2023-06-27T12:42:23.530917+0000","last_clean":"2023-06-27T12:42:23.530917+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:23.530917+0000","last_undegraded":"2023-06-27T12:42:23.530917+0000","last_fullsized":"2023-06-27T12:42:23.530917+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":110,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":651,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:23.533503+0000","last_change":"2023-06-27T12:42:22.546722+0000","last_active":"2023-06-27T12:42:23.533503+0000","last_peered":"2023-06-27T12:42:23.533503+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:23.533503+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":42880,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":406,"reported_epoch":249,"state":"active+undersized","last_fresh":"2023-06-27T12:42:22.531704+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:22.531704+0000","last_peered":"2023-06-27T12:42:22.531704+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:22.531704+0000","last_undegraded":"2023-06-27T12:42:22.531704+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":568,"reported_epoch":250,"state":"active+undersized","last_fresh":"2023-06-27T12:42:23.533316+0000","last_change":"2023-06-27T12:42:22.544551+0000","last_active":"2023-06-27T12:42:23.533316+0000","last_peered":"2023-06-27T12:42:23.533316+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:23.533316+0000","last_undegraded":"2023-06-27T12:42:23.533316+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":391,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-06-27T12:42:23.541118+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:23.541118+0000","last_peered":"2023-06-27T12:42:23.541118+0000","last_clean":"2023-06-27T12:42:23.541118+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:23.541118+0000","last_undegraded":"2023-06-27T12:42:23.541118+0000","last_fullsized":"2023-06-27T12:42:23.541118+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"247'136","reported_seq":725,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:22.535918+0000","last_change":"2023-06-27T12:42:22.535918+0000","last_active":"2023-06-27T12:42:22.535918+0000","last_peered":"2023-06-27T12:42:22.535918+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:22.535918+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3311045,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":64,"num_read_kb":63348,"num_write":88,"num_write_kb":24408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":390,"reported_epoch":250,"state":"active+undersized","last_fresh":"2023-06-27T12:42:23.542401+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:23.542401+0000","last_peered":"2023-06-27T12:42:23.542401+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:23.542401+0000","last_undegraded":"2023-06-27T12:42:23.542401+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"247'325","reported_seq":352,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:22.523736+0000","last_change":"2023-06-27T12:42:22.535286+0000","last_active":"2023-06-27T12:42:22.523736+0000","last_peered":"2023-06-27T12:42:22.523736+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:22.523736+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933036,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":31,"num_read_kb":17196,"num_write":55,"num_write_kb":16736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":387,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-06-27T12:42:23.546169+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:23.546169+0000","last_peered":"2023-06-27T12:42:23.546169+0000","last_clean":"2023-06-27T12:42:23.546169+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:23.546169+0000","last_undegraded":"2023-06-27T12:42:23.546169+0000","last_fullsized":"2023-06-27T12:42:23.546169+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":143,"reported_epoch":250,"state":"active+undersized","last_fresh":"2023-06-27T12:42:23.540691+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:23.540691+0000","last_peered":"2023-06-27T12:42:23.540691+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:23.540691+0000","last_undegraded":"2023-06-27T12:42:23.540691+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"249'299","reported_seq":922,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:23.533624+0000","last_change":"2023-06-27T12:42:22.629811+0000","last_active":"2023-06-27T12:42:23.533624+0000","last_peered":"2023-06-27T12:42:23.533624+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:23.533624+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4487667,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":267,"num_read_kb":167573,"num_write":219,"num_write_kb":63612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":391,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-06-27T12:42:23.522718+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:23.522718+0000","last_peered":"2023-06-27T12:42:23.522718+0000","last_clean":"2023-06-27T12:42:23.522718+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:23.522718+0000","last_undegraded":"2023-06-27T12:42:23.522718+0000","last_fullsized":"2023-06-27T12:42:23.522718+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"249'148","reported_seq":315,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:23.548197+0000","last_change":"2023-06-27T12:42:22.546100+0000","last_active":"2023-06-27T12:42:23.548197+0000","last_peered":"2023-06-27T12:42:23.548197+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:23.548197+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2218656,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":68,"num_read_kb":41523,"num_write":37,"num_write_kb":10230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":149,"reported_epoch":249,"state":"active+undersized","last_fresh":"2023-06-27T12:42:22.521625+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:22.521625+0000","last_peered":"2023-06-27T12:42:22.521625+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:22.521625+0000","last_undegraded":"2023-06-27T12:42:22.521625+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"230'135","reported_seq":748,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:22.693855+0000","last_change":"2023-06-27T12:42:22.537387+0000","last_active":"2023-06-27T12:42:22.693855+0000","last_peered":"2023-06-27T12:42:22.693855+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:22.693855+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6689318,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":84,"num_read_kb":96517,"num_write":96,"num_write_kb":28435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":315,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:23.617180+0000","last_change":"2023-06-27T12:42:22.548864+0000","last_active":"2023-06-27T12:42:23.617180+0000","last_peered":"2023-06-27T12:42:23.617180+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:23.617180+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":31919,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":406,"reported_epoch":249,"state":"active+undersized","last_fresh":"2023-06-27T12:42:22.528968+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:22.528968+0000","last_peered":"2023-06-27T12:42:22.528968+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:22.528968+0000","last_undegraded":"2023-06-27T12:42:22.528968+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":387,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-06-27T12:42:23.543635+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:23.543635+0000","last_peered":"2023-06-27T12:42:23.543635+0000","last_clean":"2023-06-27T12:42:23.543635+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:23.543635+0000","last_undegraded":"2023-06-27T12:42:23.543635+0000","last_fullsized":"2023-06-27T12:42:23.543635+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"249'173","reported_seq":741,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:22.564626+0000","last_change":"2023-06-27T12:42:22.531943+0000","last_active":"2023-06-27T12:42:22.564626+0000","last_peered":"2023-06-27T12:42:22.564626+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:22.564626+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6017071,"num_objects":18,"num_object_clones":15,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":80,"num_read_kb":73815,"num_write":88,"num_write_kb":25968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":75448768,"num_objects":242,"num_object_clones":198,"num_object_copies":726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1489,"num_read_kb":1049473,"num_write":1301,"num_write_kb":377185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2392,"ondisk_log_size":2392,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673230,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":179836,"kb_used_data":147968,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94192004,"statfs":{"total":96636764160,"available":96452612096,"internally_reserved":0,"allocated":151519232,"data_stored":169583511,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":192672,"kb_used_data":157096,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94179168,"statfs":{"total":96636764160,"available":96439468032,"internally_reserved":0,"allocated":160866304,"data_stored":177611100,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673230,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":208604,"kb_used_data":174468,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94163236,"statfs":{"total":96636764160,"available":96423153664,"internally_reserved":0,"allocated":178655232,"data_stored":198077431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:27.077 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:27.072+0000 7fac89bc2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:27.080 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:27.076+0000 7fac89bc2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:27.126 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:27.129 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:27.129 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:27.527 INFO:tasks.rados.rados.0.smithi138.stdout:1039: done (4 left) 2023-06-27T12:42:27.527 INFO:tasks.rados.rados.0.smithi138.stdout:1040: write oid 43 current snap is 116 2023-06-27T12:42:27.527 INFO:tasks.rados.rados.0.smithi138.stdout:1035: expect (ObjNum 374 snap 91 seq_num 374) 2023-06-27T12:42:27.648 INFO:tasks.rados.rados.0.smithi138.stdout:1034: expect (ObjNum 484 snap 113 seq_num 484) 2023-06-27T12:42:27.728 INFO:tasks.rados.rados.0.smithi138.stdout:1038: expect (ObjNum 366 snap 91 seq_num 366) 2023-06-27T12:42:27.848 INFO:tasks.rados.rados.0.smithi138.stdout:1037: finishing write tid 1 to smithi138138488-16 2023-06-27T12:42:27.849 INFO:tasks.rados.rados.0.smithi138.stdout:1040: seq_num 508 ranges {537956=705790,1644029=772160} 2023-06-27T12:42:27.855 INFO:tasks.rados.rados.0.smithi138.stdout:1040: writing smithi138138488-43 from 537956 to 1243746 tid 1 2023-06-27T12:42:27.862 INFO:tasks.rados.rados.0.smithi138.stdout:1040: writing smithi138138488-43 from 1644029 to 2416189 tid 2 2023-06-27T12:42:27.862 INFO:tasks.rados.rados.0.smithi138.stdout:1034: done (4 left) 2023-06-27T12:42:27.862 INFO:tasks.rados.rados.0.smithi138.stdout:1035: done (3 left) 2023-06-27T12:42:27.863 INFO:tasks.rados.rados.0.smithi138.stdout:1038: done (2 left) 2023-06-27T12:42:27.863 INFO:tasks.rados.rados.0.smithi138.stdout:1041: delete oid 13 current snap is 116 2023-06-27T12:42:27.863 INFO:tasks.rados.rados.0.smithi138.stdout:1037: finishing write tid 2 to smithi138138488-16 2023-06-27T12:42:27.863 INFO:tasks.rados.rados.0.smithi138.stdout:1037: finishing write tid 3 to smithi138138488-16 2023-06-27T12:42:27.863 INFO:tasks.rados.rados.0.smithi138.stdout:1037: finishing write tid 4 to smithi138138488-16 2023-06-27T12:42:27.863 INFO:tasks.rados.rados.0.smithi138.stdout:1037: oid 16 updating version 0 to 318 2023-06-27T12:42:27.863 INFO:tasks.rados.rados.0.smithi138.stdout:1037: oid 16 version 318 is already newer than 318 2023-06-27T12:42:27.864 INFO:tasks.rados.rados.0.smithi138.stdout:1037: oid 16 version 318 is already newer than 318 2023-06-27T12:42:27.864 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 318 (ObjNum 507 snap 116 seq_num 507) dirty exists 2023-06-27T12:42:27.864 INFO:tasks.rados.rados.0.smithi138.stdout:1037: left oid 16 (ObjNum 507 snap 116 seq_num 507) 2023-06-27T12:42:27.881 INFO:tasks.rados.rados.0.smithi138.stdout:1040: finishing write tid 1 to smithi138138488-43 2023-06-27T12:42:27.888 INFO:tasks.rados.rados.0.smithi138.stdout:1040: finishing write tid 2 to smithi138138488-43 2023-06-27T12:42:27.919 INFO:tasks.rados.rados.0.smithi138.stdout:1037: done (2 left) 2023-06-27T12:42:27.919 INFO:tasks.rados.rados.0.smithi138.stdout:1041: done (1 left) 2023-06-27T12:42:27.919 INFO:tasks.rados.rados.0.smithi138.stdout:1042: write oid 17 current snap is 116 2023-06-27T12:42:27.919 INFO:tasks.rados.rados.0.smithi138.stdout:1042: seq_num 510 ranges {571570=572641,1877346=694224} 2023-06-27T12:42:27.925 INFO:tasks.rados.rados.0.smithi138.stdout:1042: writing smithi138138488-17 from 571570 to 1144211 tid 1 2023-06-27T12:42:27.933 INFO:tasks.rados.rados.0.smithi138.stdout:1042: writing smithi138138488-17 from 1877346 to 2571570 tid 2 2023-06-27T12:42:27.933 INFO:tasks.rados.rados.0.smithi138.stdout:1043: write (excl) oid 5 current snap is 116 2023-06-27T12:42:27.933 INFO:tasks.rados.rados.0.smithi138.stdout:1043: seq_num 511 ranges {588377=647890,1935828=635840} 2023-06-27T12:42:27.940 INFO:tasks.rados.rados.0.smithi138.stdout:1043: writing smithi138138488-5 from 588377 to 1236267 tid 1 2023-06-27T12:42:27.948 INFO:tasks.rados.rados.0.smithi138.stdout:1043: writing smithi138138488-5 from 1935828 to 2571668 tid 2 2023-06-27T12:42:27.948 INFO:tasks.rados.rados.0.smithi138.stdout:1040: finishing write tid 3 to smithi138138488-43 2023-06-27T12:42:27.949 INFO:tasks.rados.rados.0.smithi138.stdout:1040: finishing write tid 4 to smithi138138488-43 2023-06-27T12:42:27.949 INFO:tasks.rados.rados.0.smithi138.stdout:1040: oid 43 updating version 0 to 144 2023-06-27T12:42:27.949 INFO:tasks.rados.rados.0.smithi138.stdout:1040: oid 43 version 144 is already newer than 144 2023-06-27T12:42:27.949 INFO:tasks.rados.rados.0.smithi138.stdout:1040: oid 43 version 144 is already newer than 144 2023-06-27T12:42:27.949 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 144 (ObjNum 508 snap 116 seq_num 508) dirty exists 2023-06-27T12:42:27.949 INFO:tasks.rados.rados.0.smithi138.stdout:1040: left oid 43 (ObjNum 508 snap 116 seq_num 508) 2023-06-27T12:42:27.949 INFO:tasks.rados.rados.0.smithi138.stdout:1044: read oid 22 snap 113 2023-06-27T12:42:27.950 INFO:tasks.rados.rados.0.smithi138.stdout:1044: expect (ObjNum 492 snap 113 seq_num 492) 2023-06-27T12:42:27.950 INFO:tasks.rados.rados.0.smithi138.stdout:1040: done (3 left) 2023-06-27T12:42:27.950 INFO:tasks.rados.rados.0.smithi138.stdout:1045: delete oid 8 current snap is 116 2023-06-27T12:42:27.950 INFO:tasks.rados.rados.0.smithi138.stdout:1042: finishing write tid 1 to smithi138138488-17 2023-06-27T12:42:27.954 INFO:tasks.rados.rados.0.smithi138.stdout:1042: finishing write tid 2 to smithi138138488-17 2023-06-27T12:42:27.972 INFO:tasks.rados.rados.0.smithi138.stdout:1042: finishing write tid 3 to smithi138138488-17 2023-06-27T12:42:27.999 INFO:tasks.rados.rados.0.smithi138.stdout:1045: done (3 left) 2023-06-27T12:42:27.999 INFO:tasks.rados.rados.0.smithi138.stdout:1046: snap_create 2023-06-27T12:42:28.003 INFO:tasks.rados.rados.0.smithi138.stdout:1043: finishing write tid 1 to smithi138138488-5 2023-06-27T12:42:28.012 INFO:tasks.rados.rados.0.smithi138.stdout:1042: finishing write tid 4 to smithi138138488-17 2023-06-27T12:42:28.013 INFO:tasks.rados.rados.0.smithi138.stdout:1042: oid 17 updating version 0 to 159 2023-06-27T12:42:28.013 INFO:tasks.rados.rados.0.smithi138.stdout:1042: oid 17 version 159 is already newer than 159 2023-06-27T12:42:28.013 INFO:tasks.rados.rados.0.smithi138.stdout:1042: oid 17 version 159 is already newer than 159 2023-06-27T12:42:28.013 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 159 (ObjNum 510 snap 116 seq_num 510) dirty exists 2023-06-27T12:42:28.013 INFO:tasks.rados.rados.0.smithi138.stdout:1042: left oid 17 (ObjNum 510 snap 116 seq_num 510) 2023-06-27T12:42:28.017 INFO:tasks.rados.rados.0.smithi138.stdout:1044: expect (ObjNum 492 snap 113 seq_num 492) 2023-06-27T12:42:28.143 INFO:tasks.rados.rados.0.smithi138.stdout:1043: finishing write tid 2 to smithi138138488-5 2023-06-27T12:42:28.143 INFO:tasks.rados.rados.0.smithi138.stdout:1043: finishing write tid 3 to smithi138138488-5 2023-06-27T12:42:28.143 INFO:tasks.rados.rados.0.smithi138.stdout:1043: finishing write tid 4 to smithi138138488-5 2023-06-27T12:42:28.143 INFO:tasks.rados.rados.0.smithi138.stdout:1043: oid 5 updating version 0 to 159 2023-06-27T12:42:28.144 INFO:tasks.rados.rados.0.smithi138.stdout:1043: oid 5 version 159 is already newer than 159 2023-06-27T12:42:28.144 INFO:tasks.rados.rados.0.smithi138.stdout:1043: oid 5 version 159 is already newer than 155 2023-06-27T12:42:28.144 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 5 v 159 (ObjNum 511 snap 116 seq_num 511) dirty exists 2023-06-27T12:42:28.144 INFO:tasks.rados.rados.0.smithi138.stdout:1043: left oid 5 (ObjNum 511 snap 116 seq_num 511) 2023-06-27T12:42:28.532 INFO:tasks.rados.rados.0.smithi138.stdout:1042: done (3 left) 2023-06-27T12:42:28.533 INFO:tasks.rados.rados.0.smithi138.stdout:1043: done (2 left) 2023-06-27T12:42:28.533 INFO:tasks.rados.rados.0.smithi138.stdout:1044: done (1 left) 2023-06-27T12:42:28.533 INFO:tasks.rados.rados.0.smithi138.stdout:1046: done (0 left) 2023-06-27T12:42:28.533 INFO:tasks.rados.rados.0.smithi138.stdout:1047: read oid 48 snap -1 2023-06-27T12:42:28.533 INFO:tasks.rados.rados.0.smithi138.stdout:1047: expect (ObjNum 504 snap 116 seq_num 504) 2023-06-27T12:42:28.533 INFO:tasks.rados.rados.0.smithi138.stdout:1048: write (excl) oid 45 current snap is 117 2023-06-27T12:42:28.534 INFO:tasks.rados.rados.0.smithi138.stdout:1048: seq_num 513 ranges {621991=798388,1936439=639509} 2023-06-27T12:42:28.540 INFO:tasks.rados.rados.0.smithi138.stdout:1048: writing smithi138138488-45 from 621991 to 1420379 tid 1 2023-06-27T12:42:28.548 INFO:tasks.rados.rados.0.smithi138.stdout:1048: writing smithi138138488-45 from 1936439 to 2575948 tid 2 2023-06-27T12:42:28.548 INFO:tasks.rados.rados.0.smithi138.stdout:1049: delete oid 9 current snap is 117 2023-06-27T12:42:28.574 INFO:tasks.rados.rados.0.smithi138.stdout:1047: expect (ObjNum 504 snap 116 seq_num 504) 2023-06-27T12:42:28.731 INFO:tasks.rados.rados.0.smithi138.stdout:1048: finishing write tid 1 to smithi138138488-45 2023-06-27T12:42:28.731 INFO:tasks.rados.rados.0.smithi138.stdout:1048: finishing write tid 2 to smithi138138488-45 2023-06-27T12:42:28.731 INFO:tasks.rados.rados.0.smithi138.stdout:1048: finishing write tid 3 to smithi138138488-45 2023-06-27T12:42:28.732 INFO:tasks.rados.rados.0.smithi138.stdout:1048: finishing write tid 4 to smithi138138488-45 2023-06-27T12:42:28.732 INFO:tasks.rados.rados.0.smithi138.stdout:1048: oid 45 updating version 0 to 331 2023-06-27T12:42:28.732 INFO:tasks.rados.rados.0.smithi138.stdout:1048: oid 45 version 331 is already newer than 331 2023-06-27T12:42:28.732 INFO:tasks.rados.rados.0.smithi138.stdout:1048: oid 45 version 331 is already newer than 328 2023-06-27T12:42:28.732 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 331 (ObjNum 513 snap 117 seq_num 513) dirty exists 2023-06-27T12:42:28.732 INFO:tasks.rados.rados.0.smithi138.stdout:1048: left oid 45 (ObjNum 513 snap 117 seq_num 513) 2023-06-27T12:42:28.733 INFO:tasks.rados.rados.0.smithi138.stdout:1047: done (2 left) 2023-06-27T12:42:28.733 INFO:tasks.rados.rados.0.smithi138.stdout:1048: done (1 left) 2023-06-27T12:42:28.733 INFO:tasks.rados.rados.0.smithi138.stdout:1049: done (0 left) 2023-06-27T12:42:28.733 INFO:tasks.rados.rados.0.smithi138.stdout:1050: read oid 22 snap -1 2023-06-27T12:42:28.733 INFO:tasks.rados.rados.0.smithi138.stdout:1050: expect (ObjNum 492 snap 113 seq_num 492) 2023-06-27T12:42:28.733 INFO:tasks.rados.rados.0.smithi138.stdout:1051: write oid 16 current snap is 117 2023-06-27T12:42:28.733 INFO:tasks.rados.rados.0.smithi138.stdout:1051: seq_num 515 ranges {655605=548886,1653403=643178} 2023-06-27T12:42:28.737 INFO:tasks.rados.rados.0.smithi138.stdout:1051: writing smithi138138488-16 from 655605 to 1204491 tid 1 2023-06-27T12:42:28.744 INFO:tasks.rados.rados.0.smithi138.stdout:1051: writing smithi138138488-16 from 1653403 to 2296581 tid 2 2023-06-27T12:42:28.744 INFO:tasks.rados.rados.0.smithi138.stdout:1052: read oid 7 snap 116 2023-06-27T12:42:28.744 INFO:tasks.rados.rados.0.smithi138.stdout:1052: expect deleted 2023-06-27T12:42:28.745 INFO:tasks.rados.rados.0.smithi138.stdout:1053: delete oid 10 current snap is 117 2023-06-27T12:42:28.745 INFO:tasks.rados.rados.0.smithi138.stdout:1050: expect (ObjNum 492 snap 113 seq_num 492) 2023-06-27T12:42:28.868 INFO:tasks.rados.rados.0.smithi138.stdout:1051: finishing write tid 1 to smithi138138488-16 2023-06-27T12:42:28.869 INFO:tasks.rados.rados.0.smithi138.stdout:1051: finishing write tid 2 to smithi138138488-16 2023-06-27T12:42:28.869 INFO:tasks.rados.rados.0.smithi138.stdout:1051: finishing write tid 3 to smithi138138488-16 2023-06-27T12:42:28.869 INFO:tasks.rados.rados.0.smithi138.stdout:1051: finishing write tid 4 to smithi138138488-16 2023-06-27T12:42:28.869 INFO:tasks.rados.rados.0.smithi138.stdout:1051: oid 16 updating version 0 to 322 2023-06-27T12:42:28.869 INFO:tasks.rados.rados.0.smithi138.stdout:1051: oid 16 version 322 is already newer than 322 2023-06-27T12:42:28.869 INFO:tasks.rados.rados.0.smithi138.stdout:1051: oid 16 version 322 is already newer than 322 2023-06-27T12:42:28.870 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 322 (ObjNum 515 snap 117 seq_num 515) dirty exists 2023-06-27T12:42:28.870 INFO:tasks.rados.rados.0.smithi138.stdout:1051: left oid 16 (ObjNum 515 snap 117 seq_num 515) 2023-06-27T12:42:28.870 INFO:tasks.rados.rados.0.smithi138.stdout:1050: done (3 left) 2023-06-27T12:42:28.870 INFO:tasks.rados.rados.0.smithi138.stdout:1051: done (2 left) 2023-06-27T12:42:28.870 INFO:tasks.rados.rados.0.smithi138.stdout:1052: done (1 left) 2023-06-27T12:42:28.870 INFO:tasks.rados.rados.0.smithi138.stdout:1053: done (0 left) 2023-06-27T12:42:28.870 INFO:tasks.rados.rados.0.smithi138.stdout:1054: snap_create 2023-06-27T12:42:29.131 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:29.257 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:29.252+0000 7f8449e55700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:29.260 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:29.256+0000 7f8449e55700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:29.305 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:29.308 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:29.308 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:29.533 INFO:tasks.rados.rados.0.smithi138.stdout:1054: done (0 left) 2023-06-27T12:42:29.533 INFO:tasks.rados.rados.0.smithi138.stdout:1055: write (excl) oid 38 current snap is 118 2023-06-27T12:42:29.533 INFO:tasks.rados.rados.0.smithi138.stdout:1055: seq_num 517 ranges {689219=415737,1770367=646847} 2023-06-27T12:42:29.539 INFO:tasks.rados.rados.0.smithi138.stdout:1055: writing smithi138138488-38 from 689219 to 1104956 tid 1 2023-06-27T12:42:29.548 INFO:tasks.rados.rados.0.smithi138.stdout:1055: writing smithi138138488-38 from 1770367 to 2417214 tid 2 2023-06-27T12:42:29.548 INFO:tasks.rados.rados.0.smithi138.stdout:1056: read oid 25 snap -1 2023-06-27T12:42:29.548 INFO:tasks.rados.rados.0.smithi138.stdout:1056: expect (ObjNum 501 snap 114 seq_num 501) 2023-06-27T12:42:29.548 INFO:tasks.rados.rados.0.smithi138.stdout:1057: read oid 17 snap -1 2023-06-27T12:42:29.549 INFO:tasks.rados.rados.0.smithi138.stdout:1057: expect (ObjNum 510 snap 116 seq_num 510) 2023-06-27T12:42:29.549 INFO:tasks.rados.rados.0.smithi138.stdout:1058: read oid 43 snap -1 2023-06-27T12:42:29.549 INFO:tasks.rados.rados.0.smithi138.stdout:1058: expect (ObjNum 508 snap 116 seq_num 508) 2023-06-27T12:42:29.549 INFO:tasks.rados.rados.0.smithi138.stdout:1059: read oid 18 snap 116 2023-06-27T12:42:29.549 INFO:tasks.rados.rados.0.smithi138.stdout:1059: expect (ObjNum 470 snap 110 seq_num 470) 2023-06-27T12:42:29.549 INFO:tasks.rados.rados.0.smithi138.stdout:1060: snap_create 2023-06-27T12:42:29.590 INFO:tasks.rados.rados.0.smithi138.stdout:1056: expect (ObjNum 501 snap 114 seq_num 501) 2023-06-27T12:42:29.686 INFO:tasks.rados.rados.0.smithi138.stdout:1057: expect (ObjNum 510 snap 116 seq_num 510) 2023-06-27T12:42:29.783 INFO:tasks.rados.rados.0.smithi138.stdout:1058: expect (ObjNum 508 snap 116 seq_num 508) 2023-06-27T12:42:29.875 INFO:tasks.rados.rados.0.smithi138.stdout:1059: expect (ObjNum 470 snap 110 seq_num 470) 2023-06-27T12:42:30.024 INFO:tasks.rados.rados.0.smithi138.stdout:1055: finishing write tid 1 to smithi138138488-38 2023-06-27T12:42:30.025 INFO:tasks.rados.rados.0.smithi138.stdout:1055: finishing write tid 2 to smithi138138488-38 2023-06-27T12:42:30.025 INFO:tasks.rados.rados.0.smithi138.stdout:1055: finishing write tid 3 to smithi138138488-38 2023-06-27T12:42:30.025 INFO:tasks.rados.rados.0.smithi138.stdout:1055: finishing write tid 4 to smithi138138488-38 2023-06-27T12:42:30.025 INFO:tasks.rados.rados.0.smithi138.stdout:1055: oid 38 updating version 0 to 32 2023-06-27T12:42:30.025 INFO:tasks.rados.rados.0.smithi138.stdout:1055: oid 38 version 32 is already newer than 30 2023-06-27T12:42:30.025 INFO:tasks.rados.rados.0.smithi138.stdout:1055: oid 38 version 32 is already newer than 32 2023-06-27T12:42:30.026 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 32 (ObjNum 517 snap 118 seq_num 517) dirty exists 2023-06-27T12:42:30.026 INFO:tasks.rados.rados.0.smithi138.stdout:1055: left oid 38 (ObjNum 517 snap 118 seq_num 517) 2023-06-27T12:42:30.543 INFO:tasks.rados.rados.0.smithi138.stdout:1055: done (5 left) 2023-06-27T12:42:30.544 INFO:tasks.rados.rados.0.smithi138.stdout:1056: done (4 left) 2023-06-27T12:42:30.544 INFO:tasks.rados.rados.0.smithi138.stdout:1057: done (3 left) 2023-06-27T12:42:30.544 INFO:tasks.rados.rados.0.smithi138.stdout:1058: done (2 left) 2023-06-27T12:42:30.544 INFO:tasks.rados.rados.0.smithi138.stdout:1059: done (1 left) 2023-06-27T12:42:30.544 INFO:tasks.rados.rados.0.smithi138.stdout:1060: done (0 left) 2023-06-27T12:42:30.544 INFO:tasks.rados.rados.0.smithi138.stdout:1061: snap_create 2023-06-27T12:42:30.895 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~116s 2023-06-27T12:42:31.309 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:31.442 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:31.436+0000 7f7b51680700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:31.444 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:31.441+0000 7f7b51680700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:31.492 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:31.493 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:31.545 INFO:tasks.rados.rados.0.smithi138.stdout:1061: done (0 left) 2023-06-27T12:42:31.546 INFO:tasks.rados.rados.0.smithi138.stdout:1062: read oid 24 snap 116 2023-06-27T12:42:31.546 INFO:tasks.rados.rados.0.smithi138.stdout:1062: expect (ObjNum 471 snap 110 seq_num 471) 2023-06-27T12:42:31.546 INFO:tasks.rados.rados.0.smithi138.stdout:1063: snap_create 2023-06-27T12:42:31.611 INFO:tasks.rados.rados.0.smithi138.stdout:1062: expect (ObjNum 471 snap 110 seq_num 471) 2023-06-27T12:42:31.991 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:42:31.992 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:42:31.993 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:32.120 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:32.112+0000 7f8289cb9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:32.122 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:32.119+0000 7f8289cb9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:32.316 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:32.318 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:32.329 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2023-06-27T12:42:31.964921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80323161,"num_objects":250,"num_object_clones":205,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1527,"num_read_kb":1077194,"num_write":1337,"num_write_kb":388189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2539,"ondisk_log_size":2539,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":782268,"kb_used_data":654492,"kb_used_omap":0,"kb_used_meta":127744,"kb_avail":376705092,"statfs":{"total":386547056640,"available":385746014208,"internally_reserved":0,"allocated":670199808,"data_stored":750984494,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":130809856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4874393,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":38,"num_read_kb":27721,"num_write":36,"num_write_kb":11004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001243"},"pg_stats":[{"pgid":"3.f","version":"248'29","reported_seq":604,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.617089+0000","last_change":"2023-06-27T12:42:27.820471+0000","last_active":"2023-06-27T12:42:28.617089+0000","last_peered":"2023-06-27T12:42:28.617089+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:28.617089+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":8,"num_read_kb":4,"num_write":19,"num_write_kb":5014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"254'159","reported_seq":802,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.553035+0000","last_change":"2023-06-27T12:42:27.547394+0000","last_active":"2023-06-27T12:42:28.553035+0000","last_peered":"2023-06-27T12:42:28.553035+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:28.553035+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12000528,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":130148,"num_write":114,"num_write_kb":33537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"250'187","reported_seq":793,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.546548+0000","last_change":"2023-06-27T12:42:27.980525+0000","last_active":"2023-06-27T12:42:28.546548+0000","last_peered":"2023-06-27T12:42:28.546548+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:28.546548+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13210698,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":68222,"num_write":137,"num_write_kb":39629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"230'100","reported_seq":737,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.601939+0000","last_change":"2023-06-27T12:42:27.822812+0000","last_active":"2023-06-27T12:42:28.601939+0000","last_peered":"2023-06-27T12:42:28.601939+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:28.601939+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9865907,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":96,"num_read_kb":54526,"num_write":79,"num_write_kb":22429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"254'69","reported_seq":308,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.610093+0000","last_change":"2023-06-27T12:42:27.839157+0000","last_active":"2023-06-27T12:42:28.610093+0000","last_peered":"2023-06-27T12:42:28.610093+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:28.610093+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":68,"num_read_kb":34135,"num_write":29,"num_write_kb":8919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"255'322","reported_seq":818,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-06-27T12:42:28.758615+0000","last_change":"2023-06-27T12:42:27.839888+0000","last_active":"2023-06-27T12:42:28.758615+0000","last_peered":"2023-06-27T12:42:28.758615+0000","last_clean":"2023-06-27T12:42:28.758615+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:28.758615+0000","last_undegraded":"2023-06-27T12:42:28.758615+0000","last_fullsized":"2023-06-27T12:42:28.758615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8605448,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":247,"num_read_kb":149315,"num_write":192,"num_write_kb":55215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"249'154","reported_seq":658,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-06-27T12:42:28.610038+0000","last_change":"2023-06-27T12:42:27.822555+0000","last_active":"2023-06-27T12:42:28.610038+0000","last_peered":"2023-06-27T12:42:28.610038+0000","last_clean":"2023-06-27T12:42:28.610038+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:28.610038+0000","last_undegraded":"2023-06-27T12:42:28.610038+0000","last_fullsized":"2023-06-27T12:42:28.610038+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":110,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":664,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.545242+0000","last_change":"2023-06-27T12:42:27.971745+0000","last_active":"2023-06-27T12:42:28.545242+0000","last_peered":"2023-06-27T12:42:28.545242+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:28.545242+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":416,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-06-27T12:42:28.548241+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:28.548241+0000","last_peered":"2023-06-27T12:42:28.548241+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:28.548241+0000","last_undegraded":"2023-06-27T12:42:28.548241+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":578,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-06-27T12:42:28.551421+0000","last_change":"2023-06-27T12:42:27.980976+0000","last_active":"2023-06-27T12:42:28.551421+0000","last_peered":"2023-06-27T12:42:28.551421+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:28.551421+0000","last_undegraded":"2023-06-27T12:42:28.551421+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":399,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-06-27T12:42:28.604992+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:28.604992+0000","last_peered":"2023-06-27T12:42:28.604992+0000","last_clean":"2023-06-27T12:42:28.604992+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:28.604992+0000","last_undegraded":"2023-06-27T12:42:28.604992+0000","last_fullsized":"2023-06-27T12:42:28.604992+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"251'139","reported_seq":741,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.549470+0000","last_change":"2023-06-27T12:42:27.549447+0000","last_active":"2023-06-27T12:42:28.549470+0000","last_peered":"2023-06-27T12:42:28.549470+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:28.549470+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5764966,"num_objects":16,"num_object_clones":11,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":65,"num_read_kb":63349,"num_write":92,"num_write_kb":25728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":16}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":398,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-06-27T12:42:28.549379+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:28.549379+0000","last_peered":"2023-06-27T12:42:28.549379+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:28.549379+0000","last_undegraded":"2023-06-27T12:42:28.549379+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"255'331","reported_seq":373,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.624108+0000","last_change":"2023-06-27T12:42:27.550212+0000","last_active":"2023-06-27T12:42:28.624108+0000","last_peered":"2023-06-27T12:42:28.624108+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:28.624108+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621991,"num_objects":11,"num_object_clones":10,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":38,"num_read_kb":24501,"num_write":59,"num_write_kb":18141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":11}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":395,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-06-27T12:42:28.557279+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:28.557279+0000","last_peered":"2023-06-27T12:42:28.557279+0000","last_clean":"2023-06-27T12:42:28.557279+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:28.557279+0000","last_undegraded":"2023-06-27T12:42:28.557279+0000","last_fullsized":"2023-06-27T12:42:28.557279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":151,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-06-27T12:42:28.613160+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:28.613160+0000","last_peered":"2023-06-27T12:42:28.613160+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:28.613160+0000","last_undegraded":"2023-06-27T12:42:28.613160+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"250'301","reported_seq":933,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.558486+0000","last_change":"2023-06-27T12:42:27.970713+0000","last_active":"2023-06-27T12:42:28.558486+0000","last_peered":"2023-06-27T12:42:28.558486+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:28.558486+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2386693,"num_objects":41,"num_object_clones":35,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":267,"num_read_kb":167573,"num_write":219,"num_write_kb":63612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":41}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":399,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-06-27T12:42:28.603942+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:28.603942+0000","last_peered":"2023-06-27T12:42:28.603942+0000","last_clean":"2023-06-27T12:42:28.603942+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:28.603942+0000","last_undegraded":"2023-06-27T12:42:28.603942+0000","last_fullsized":"2023-06-27T12:42:28.603942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"254'159","reported_seq":333,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.553501+0000","last_change":"2023-06-27T12:42:28.041885+0000","last_active":"2023-06-27T12:42:28.553501+0000","last_peered":"2023-06-27T12:42:28.553501+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:28.553501+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5059138,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":71,"num_read_kb":41525,"num_write":45,"num_write_kb":12564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-06-27T12:42:28.551726+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:28.551726+0000","last_peered":"2023-06-27T12:42:28.551726+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:28.551726+0000","last_undegraded":"2023-06-27T12:42:28.551726+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"254'144","reported_seq":764,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.552580+0000","last_change":"2023-06-27T12:42:27.548816+0000","last_active":"2023-06-27T12:42:28.552580+0000","last_peered":"2023-06-27T12:42:28.552580+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:28.552580+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9227274,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":85,"num_read_kb":96518,"num_write":100,"num_write_kb":29880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":328,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.559194+0000","last_change":"2023-06-27T12:42:27.970176+0000","last_active":"2023-06-27T12:42:28.559194+0000","last_peered":"2023-06-27T12:42:28.559194+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:28.559194+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":36964,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":416,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-06-27T12:42:28.549099+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:28.549099+0000","last_peered":"2023-06-27T12:42:28.549099+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:28.549099+0000","last_undegraded":"2023-06-27T12:42:28.549099+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":395,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-06-27T12:42:28.550395+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:28.550395+0000","last_peered":"2023-06-27T12:42:28.550395+0000","last_clean":"2023-06-27T12:42:28.550395+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:28.550395+0000","last_undegraded":"2023-06-27T12:42:28.550395+0000","last_fullsized":"2023-06-27T12:42:28.550395+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"252'177","reported_seq":763,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:28.732715+0000","last_change":"2023-06-27T12:42:27.592604+0000","last_active":"2023-06-27T12:42:28.732715+0000","last_peered":"2023-06-27T12:42:28.732715+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:28.732715+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739772,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":87,"num_read_kb":84932,"num_write":92,"num_write_kb":26834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":80323161,"num_objects":250,"num_object_clones":205,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1527,"num_read_kb":1077194,"num_write":1337,"num_write_kb":388189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2442,"ondisk_log_size":2442,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673235,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":187628,"kb_used_data":156708,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94184212,"statfs":{"total":96636764160,"available":96444633088,"internally_reserved":0,"allocated":160468992,"data_stored":179801566,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":195568,"kb_used_data":164648,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94176272,"statfs":{"total":96636764160,"available":96436502528,"internally_reserved":0,"allocated":168599552,"data_stored":185879793,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673235,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":217792,"kb_used_data":182776,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94154048,"statfs":{"total":96636764160,"available":96413745152,"internally_reserved":0,"allocated":187162624,"data_stored":208263583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:32.331 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.3 2023-06-27T12:42:32.460 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:32.451+0000 7f229263d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:32.459+0000 7f229263d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:32.544 INFO:tasks.rados.rados.0.smithi138.stdout:1062: done (1 left) 2023-06-27T12:42:32.544 INFO:tasks.rados.rados.0.smithi138.stdout:1063: done (0 left) 2023-06-27T12:42:32.545 INFO:tasks.rados.rados.0.smithi138.stdout:1064: write (excl) oid 37 current snap is 121 2023-06-27T12:42:32.545 INFO:tasks.rados.rados.0.smithi138.stdout:1064: seq_num 518 ranges {706026=490986,1828849=790505} 2023-06-27T12:42:32.553 INFO:tasks.rados.rados.0.smithi138.stdout:1064: writing smithi138138488-37 from 706026 to 1197012 tid 1 2023-06-27T12:42:32.564 INFO:tasks.rados.rados.0.smithi138.stdout:1064: writing smithi138138488-37 from 1828849 to 2619354 tid 2 2023-06-27T12:42:32.565 INFO:tasks.rados.rados.0.smithi138.stdout:1065: read oid 22 snap -1 2023-06-27T12:42:32.565 INFO:tasks.rados.rados.0.smithi138.stdout:1065: expect (ObjNum 492 snap 113 seq_num 492) 2023-06-27T12:42:32.565 INFO:tasks.rados.rados.0.smithi138.stdout:1066: snap_remove snap 117 2023-06-27T12:42:32.686 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.3] on osd.2 to force-backfill; 2023-06-27T12:42:32.686 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:42:33.494 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:33.555 INFO:tasks.rados.rados.0.smithi138.stdout:1066: done (2 left) 2023-06-27T12:42:33.556 INFO:tasks.rados.rados.0.smithi138.stdout:1067: read oid 15 snap -1 2023-06-27T12:42:33.556 INFO:tasks.rados.rados.0.smithi138.stdout:1067: expect deleted 2023-06-27T12:42:33.556 INFO:tasks.rados.rados.0.smithi138.stdout:1068: write (excl) oid 16 current snap is 121 2023-06-27T12:42:33.556 INFO:tasks.rados.rados.0.smithi138.stdout:1068: seq_num 519 ranges {722833=566235,1887331=650516} 2023-06-27T12:42:33.565 INFO:tasks.rados.rados.0.smithi138.stdout:1068: writing smithi138138488-16 from 722833 to 1289068 tid 1 2023-06-27T12:42:33.575 INFO:tasks.rados.rados.0.smithi138.stdout:1068: writing smithi138138488-16 from 1887331 to 2537847 tid 2 2023-06-27T12:42:33.575 INFO:tasks.rados.rados.0.smithi138.stdout:1069: read oid 6 snap -1 2023-06-27T12:42:33.575 INFO:tasks.rados.rados.0.smithi138.stdout:1069: expect (ObjNum 435 snap 100 seq_num 435) 2023-06-27T12:42:33.575 INFO:tasks.rados.rados.0.smithi138.stdout:1070: write (excl) oid 30 current snap is 121 2023-06-27T12:42:33.576 INFO:tasks.rados.rados.0.smithi138.stdout:1070: seq_num 520 ranges {739640=641484,1829460=794174} 2023-06-27T12:42:33.585 INFO:tasks.rados.rados.0.smithi138.stdout:1070: writing smithi138138488-30 from 739640 to 1381124 tid 1 2023-06-27T12:42:33.596 INFO:tasks.rados.rados.0.smithi138.stdout:1070: writing smithi138138488-30 from 1829460 to 2623634 tid 2 2023-06-27T12:42:33.596 INFO:tasks.rados.rados.0.smithi138.stdout:1071: write oid 23 current snap is 121 2023-06-27T12:42:33.596 INFO:tasks.rados.rados.0.smithi138.stdout:1071: seq_num 521 ranges {756447=716733,1887942=537832} 2023-06-27T12:42:33.607 INFO:tasks.rados.rados.0.smithi138.stdout:1071: writing smithi138138488-23 from 756447 to 1473180 tid 1 2023-06-27T12:42:33.615 INFO:tasks.rados.rados.0.smithi138.stdout:1071: writing smithi138138488-23 from 1887942 to 2425774 tid 2 2023-06-27T12:42:33.615 INFO:tasks.rados.rados.0.smithi138.stdout:1072: write oid 43 current snap is 121 2023-06-27T12:42:33.616 INFO:tasks.rados.rados.0.smithi138.stdout:1065: expect (ObjNum 492 snap 113 seq_num 492) 2023-06-27T12:42:33.641 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:33.635+0000 7f7da660f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:33.645 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:33.641+0000 7f7da660f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:33.746 INFO:tasks.rados.rados.0.smithi138.stdout:1064: finishing write tid 1 to smithi138138488-37 2023-06-27T12:42:33.747 INFO:tasks.rados.rados.0.smithi138.stdout:1064: finishing write tid 2 to smithi138138488-37 2023-06-27T12:42:33.747 INFO:tasks.rados.rados.0.smithi138.stdout:1072: seq_num 522 ranges {773254=791982,2346424=426830} 2023-06-27T12:42:33.759 INFO:tasks.rados.rados.0.smithi138.stdout:1072: writing smithi138138488-43 from 773254 to 1565236 tid 1 2023-06-27T12:42:33.768 INFO:tasks.rados.rados.0.smithi138.stdout:1072: writing smithi138138488-43 from 2346424 to 2773254 tid 2 2023-06-27T12:42:33.769 INFO:tasks.rados.rados.0.smithi138.stdout:1065: done (7 left) 2023-06-27T12:42:33.769 INFO:tasks.rados.rados.0.smithi138.stdout:1073: write oid 12 current snap is 121 2023-06-27T12:42:33.769 INFO:tasks.rados.rados.0.smithi138.stdout:1064: finishing write tid 3 to smithi138138488-37 2023-06-27T12:42:33.769 INFO:tasks.rados.rados.0.smithi138.stdout:1064: finishing write tid 4 to smithi138138488-37 2023-06-27T12:42:33.770 INFO:tasks.rados.rados.0.smithi138.stdout:1064: oid 37 updating version 0 to 188 2023-06-27T12:42:33.770 INFO:tasks.rados.rados.0.smithi138.stdout:1064: oid 37 updating version 188 to 191 2023-06-27T12:42:33.770 INFO:tasks.rados.rados.0.smithi138.stdout:1064: oid 37 version 191 is already newer than 191 2023-06-27T12:42:33.770 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 191 (ObjNum 518 snap 121 seq_num 518) dirty exists 2023-06-27T12:42:33.770 INFO:tasks.rados.rados.0.smithi138.stdout:1064: left oid 37 (ObjNum 518 snap 121 seq_num 518) 2023-06-27T12:42:33.770 INFO:tasks.rados.rados.0.smithi138.stdout:1071: finishing write tid 1 to smithi138138488-23 2023-06-27T12:42:33.770 INFO:tasks.rados.rados.0.smithi138.stdout:1069: expect (ObjNum 435 snap 100 seq_num 435) 2023-06-27T12:42:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:33.786 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:33.787 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:33.894 INFO:tasks.rados.rados.0.smithi138.stdout:1068: finishing write tid 1 to smithi138138488-16 2023-06-27T12:42:33.894 INFO:tasks.rados.rados.0.smithi138.stdout:1071: finishing write tid 2 to smithi138138488-23 2023-06-27T12:42:33.894 INFO:tasks.rados.rados.0.smithi138.stdout:1071: finishing write tid 3 to smithi138138488-23 2023-06-27T12:42:33.895 INFO:tasks.rados.rados.0.smithi138.stdout:1071: finishing write tid 4 to smithi138138488-23 2023-06-27T12:42:33.895 INFO:tasks.rados.rados.0.smithi138.stdout:1071: oid 23 updating version 0 to 143 2023-06-27T12:42:33.895 INFO:tasks.rados.rados.0.smithi138.stdout:1071: oid 23 version 143 is already newer than 143 2023-06-27T12:42:33.895 INFO:tasks.rados.rados.0.smithi138.stdout:1071: oid 23 version 143 is already newer than 143 2023-06-27T12:42:33.895 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 143 (ObjNum 521 snap 121 seq_num 521) dirty exists 2023-06-27T12:42:33.895 INFO:tasks.rados.rados.0.smithi138.stdout:1071: left oid 23 (ObjNum 521 snap 121 seq_num 521) 2023-06-27T12:42:33.895 INFO:tasks.rados.rados.0.smithi138.stdout:1070: finishing write tid 1 to smithi138138488-30 2023-06-27T12:42:33.896 INFO:tasks.rados.rados.0.smithi138.stdout:1068: finishing write tid 2 to smithi138138488-16 2023-06-27T12:42:33.896 INFO:tasks.rados.rados.0.smithi138.stdout:1068: finishing write tid 3 to smithi138138488-16 2023-06-27T12:42:33.896 INFO:tasks.rados.rados.0.smithi138.stdout:1068: finishing write tid 4 to smithi138138488-16 2023-06-27T12:42:33.896 INFO:tasks.rados.rados.0.smithi138.stdout:1068: oid 16 updating version 0 to 323 2023-06-27T12:42:33.896 INFO:tasks.rados.rados.0.smithi138.stdout:1068: oid 16 updating version 323 to 326 2023-06-27T12:42:33.896 INFO:tasks.rados.rados.0.smithi138.stdout:1068: oid 16 version 326 is already newer than 326 2023-06-27T12:42:33.897 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 326 (ObjNum 519 snap 121 seq_num 519) dirty exists 2023-06-27T12:42:33.897 INFO:tasks.rados.rados.0.smithi138.stdout:1068: left oid 16 (ObjNum 519 snap 121 seq_num 519) 2023-06-27T12:42:33.897 INFO:tasks.rados.rados.0.smithi138.stdout:1073: seq_num 523 ranges {790061=467231,2004906=541501} 2023-06-27T12:42:33.900 INFO:tasks.rados.rados.0.smithi138.stdout:1073: writing smithi138138488-12 from 790061 to 1257292 tid 1 2023-06-27T12:42:33.910 INFO:tasks.rados.rados.0.smithi138.stdout:1073: writing smithi138138488-12 from 2004906 to 2546407 tid 2 2023-06-27T12:42:33.910 INFO:tasks.rados.rados.0.smithi138.stdout:1064: done (7 left) 2023-06-27T12:42:33.910 INFO:tasks.rados.rados.0.smithi138.stdout:1067: done (6 left) 2023-06-27T12:42:33.910 INFO:tasks.rados.rados.0.smithi138.stdout:1068: done (5 left) 2023-06-27T12:42:33.910 INFO:tasks.rados.rados.0.smithi138.stdout:1069: done (4 left) 2023-06-27T12:42:33.911 INFO:tasks.rados.rados.0.smithi138.stdout:1071: done (3 left) 2023-06-27T12:42:33.911 INFO:tasks.rados.rados.0.smithi138.stdout:1074: snap_remove snap 119 2023-06-27T12:42:34.572 INFO:tasks.rados.rados.0.smithi138.stdout:1074: done (3 left) 2023-06-27T12:42:34.573 INFO:tasks.rados.rados.0.smithi138.stdout:1075: write oid 6 current snap is 121 2023-06-27T12:42:34.573 INFO:tasks.rados.rados.0.smithi138.stdout:1072: finishing write tid 1 to smithi138138488-43 2023-06-27T12:42:34.573 INFO:tasks.rados.rados.0.smithi138.stdout:1075: seq_num 524 ranges {406868=542480,1547035=401512,2690354=116514} 2023-06-27T12:42:34.579 INFO:tasks.rados.rados.0.smithi138.stdout:1075: writing smithi138138488-6 from 406868 to 949348 tid 1 2023-06-27T12:42:34.587 INFO:tasks.rados.rados.0.smithi138.stdout:1075: writing smithi138138488-6 from 1547035 to 1948547 tid 2 2023-06-27T12:42:34.592 INFO:tasks.rados.rados.0.smithi138.stdout:1075: writing smithi138138488-6 from 2690354 to 2806868 tid 3 2023-06-27T12:42:34.593 INFO:tasks.rados.rados.0.smithi138.stdout:1076: write (excl) oid 31 current snap is 121 2023-06-27T12:42:34.593 INFO:tasks.rados.rados.0.smithi138.stdout:1076: seq_num 525 ranges {423675=734082,1721870=545170,2717777=105898} 2023-06-27T12:42:34.602 INFO:tasks.rados.rados.0.smithi138.stdout:1076: writing smithi138138488-31 from 423675 to 1157757 tid 1 2023-06-27T12:42:34.610 INFO:tasks.rados.rados.0.smithi138.stdout:1076: writing smithi138138488-31 from 1721870 to 2267040 tid 2 2023-06-27T12:42:34.614 INFO:tasks.rados.rados.0.smithi138.stdout:1076: writing smithi138138488-31 from 2717777 to 2823675 tid 3 2023-06-27T12:42:34.615 INFO:tasks.rados.rados.0.smithi138.stdout:1077: read oid 20 snap -1 2023-06-27T12:42:34.615 INFO:tasks.rados.rados.0.smithi138.stdout:1077: expect (ObjNum 479 snap 112 seq_num 479) 2023-06-27T12:42:34.615 INFO:tasks.rados.rados.0.smithi138.stdout:1078: read oid 37 snap -1 2023-06-27T12:42:34.615 INFO:tasks.rados.rados.0.smithi138.stdout:1078: expect (ObjNum 518 snap 121 seq_num 518) 2023-06-27T12:42:34.615 INFO:tasks.rados.rados.0.smithi138.stdout:1079: delete oid 5 current snap is 121 2023-06-27T12:42:34.615 INFO:tasks.rados.rados.0.smithi138.stdout:1070: finishing write tid 2 to smithi138138488-30 2023-06-27T12:42:34.616 INFO:tasks.rados.rados.0.smithi138.stdout:1070: finishing write tid 3 to smithi138138488-30 2023-06-27T12:42:34.616 INFO:tasks.rados.rados.0.smithi138.stdout:1070: finishing write tid 4 to smithi138138488-30 2023-06-27T12:42:34.616 INFO:tasks.rados.rados.0.smithi138.stdout:1070: oid 30 updating version 0 to 163 2023-06-27T12:42:34.616 INFO:tasks.rados.rados.0.smithi138.stdout:1070: oid 30 version 163 is already newer than 163 2023-06-27T12:42:34.616 INFO:tasks.rados.rados.0.smithi138.stdout:1070: oid 30 version 163 is already newer than 160 2023-06-27T12:42:34.616 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 163 (ObjNum 520 snap 121 seq_num 520) dirty exists 2023-06-27T12:42:34.616 INFO:tasks.rados.rados.0.smithi138.stdout:1070: left oid 30 (ObjNum 520 snap 121 seq_num 520) 2023-06-27T12:42:34.617 INFO:tasks.rados.rados.0.smithi138.stdout:1072: finishing write tid 2 to smithi138138488-43 2023-06-27T12:42:34.617 INFO:tasks.rados.rados.0.smithi138.stdout:1072: finishing write tid 3 to smithi138138488-43 2023-06-27T12:42:34.617 INFO:tasks.rados.rados.0.smithi138.stdout:1072: finishing write tid 4 to smithi138138488-43 2023-06-27T12:42:34.617 INFO:tasks.rados.rados.0.smithi138.stdout:1072: oid 43 updating version 0 to 147 2023-06-27T12:42:34.618 INFO:tasks.rados.rados.0.smithi138.stdout:1072: oid 43 updating version 147 to 148 2023-06-27T12:42:34.618 INFO:tasks.rados.rados.0.smithi138.stdout:1072: oid 43 version 148 is already newer than 148 2023-06-27T12:42:34.618 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 148 (ObjNum 522 snap 121 seq_num 522) dirty exists 2023-06-27T12:42:34.618 INFO:tasks.rados.rados.0.smithi138.stdout:1072: left oid 43 (ObjNum 522 snap 121 seq_num 522) 2023-06-27T12:42:34.618 INFO:tasks.rados.rados.0.smithi138.stdout:1073: finishing write tid 1 to smithi138138488-12 2023-06-27T12:42:34.618 INFO:tasks.rados.rados.0.smithi138.stdout:1073: finishing write tid 2 to smithi138138488-12 2023-06-27T12:42:34.618 INFO:tasks.rados.rados.0.smithi138.stdout:1073: finishing write tid 3 to smithi138138488-12 2023-06-27T12:42:34.618 INFO:tasks.rados.rados.0.smithi138.stdout:1073: finishing write tid 4 to smithi138138488-12 2023-06-27T12:42:34.619 INFO:tasks.rados.rados.0.smithi138.stdout:1073: oid 12 updating version 0 to 197 2023-06-27T12:42:34.619 INFO:tasks.rados.rados.0.smithi138.stdout:1073: oid 12 version 197 is already newer than 197 2023-06-27T12:42:34.619 INFO:tasks.rados.rados.0.smithi138.stdout:1073: oid 12 version 197 is already newer than 197 2023-06-27T12:42:34.619 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 197 (ObjNum 523 snap 121 seq_num 523) dirty exists 2023-06-27T12:42:34.619 INFO:tasks.rados.rados.0.smithi138.stdout:1073: left oid 12 (ObjNum 523 snap 121 seq_num 523) 2023-06-27T12:42:34.651 INFO:tasks.rados.rados.0.smithi138.stdout:1077: expect (ObjNum 479 snap 112 seq_num 479) 2023-06-27T12:42:34.757 INFO:tasks.rados.rados.0.smithi138.stdout:1078: expect (ObjNum 518 snap 121 seq_num 518) 2023-06-27T12:42:34.903 INFO:tasks.rados.rados.0.smithi138.stdout:1075: finishing write tid 1 to smithi138138488-6 2023-06-27T12:42:34.903 INFO:tasks.rados.rados.0.smithi138.stdout:1075: finishing write tid 2 to smithi138138488-6 2023-06-27T12:42:34.903 INFO:tasks.rados.rados.0.smithi138.stdout:1075: finishing write tid 3 to smithi138138488-6 2023-06-27T12:42:34.903 INFO:tasks.rados.rados.0.smithi138.stdout:1075: finishing write tid 4 to smithi138138488-6 2023-06-27T12:42:34.904 INFO:tasks.rados.rados.0.smithi138.stdout:1075: finishing write tid 5 to smithi138138488-6 2023-06-27T12:42:34.904 INFO:tasks.rados.rados.0.smithi138.stdout:1075: oid 6 updating version 0 to 150 2023-06-27T12:42:34.904 INFO:tasks.rados.rados.0.smithi138.stdout:1075: oid 6 version 150 is already newer than 150 2023-06-27T12:42:34.904 INFO:tasks.rados.rados.0.smithi138.stdout:1075: oid 6 version 150 is already newer than 150 2023-06-27T12:42:34.904 INFO:tasks.rados.rados.0.smithi138.stdout:1075: oid 6 version 150 is already newer than 150 2023-06-27T12:42:34.904 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 150 (ObjNum 524 snap 121 seq_num 524) dirty exists 2023-06-27T12:42:34.904 INFO:tasks.rados.rados.0.smithi138.stdout:1075: left oid 6 (ObjNum 524 snap 121 seq_num 524) 2023-06-27T12:42:34.905 INFO:tasks.rados.rados.0.smithi138.stdout:1076: finishing write tid 1 to smithi138138488-31 2023-06-27T12:42:34.905 INFO:tasks.rados.rados.0.smithi138.stdout:1070: done (7 left) 2023-06-27T12:42:34.905 INFO:tasks.rados.rados.0.smithi138.stdout:1072: done (6 left) 2023-06-27T12:42:34.905 INFO:tasks.rados.rados.0.smithi138.stdout:1073: done (5 left) 2023-06-27T12:42:34.905 INFO:tasks.rados.rados.0.smithi138.stdout:1075: done (4 left) 2023-06-27T12:42:34.905 INFO:tasks.rados.rados.0.smithi138.stdout:1077: done (3 left) 2023-06-27T12:42:34.905 INFO:tasks.rados.rados.0.smithi138.stdout:1078: done (2 left) 2023-06-27T12:42:34.906 INFO:tasks.rados.rados.0.smithi138.stdout:1079: done (1 left) 2023-06-27T12:42:34.906 INFO:tasks.rados.rados.0.smithi138.stdout:1080: snap_create 2023-06-27T12:42:34.906 INFO:tasks.rados.rados.0.smithi138.stdout:1076: finishing write tid 2 to smithi138138488-31 2023-06-27T12:42:34.906 INFO:tasks.rados.rados.0.smithi138.stdout:1076: finishing write tid 3 to smithi138138488-31 2023-06-27T12:42:34.906 INFO:tasks.rados.rados.0.smithi138.stdout:1076: finishing write tid 4 to smithi138138488-31 2023-06-27T12:42:34.906 INFO:tasks.rados.rados.0.smithi138.stdout:1076: finishing write tid 5 to smithi138138488-31 2023-06-27T12:42:34.906 INFO:tasks.rados.rados.0.smithi138.stdout:1076: oid 31 updating version 0 to 332 2023-06-27T12:42:34.906 INFO:tasks.rados.rados.0.smithi138.stdout:1076: oid 31 version 332 is already newer than 332 2023-06-27T12:42:34.907 INFO:tasks.rados.rados.0.smithi138.stdout:1076: oid 31 version 332 is already newer than 329 2023-06-27T12:42:34.907 INFO:tasks.rados.rados.0.smithi138.stdout:1076: oid 31 version 332 is already newer than 332 2023-06-27T12:42:34.907 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 332 (ObjNum 525 snap 121 seq_num 525) dirty exists 2023-06-27T12:42:34.907 INFO:tasks.rados.rados.0.smithi138.stdout:1076: left oid 31 (ObjNum 525 snap 121 seq_num 525) 2023-06-27T12:42:35.432 INFO:tasks.rados.rados.0.smithi138.stdout:1076: done (1 left) 2023-06-27T12:42:35.432 INFO:tasks.rados.rados.0.smithi138.stdout:1080: done (0 left) 2023-06-27T12:42:35.433 INFO:tasks.rados.rados.0.smithi138.stdout:1081: delete oid 17 current snap is 122 2023-06-27T12:42:35.468 INFO:tasks.rados.rados.0.smithi138.stdout:1081: done (0 left) 2023-06-27T12:42:35.468 INFO:tasks.rados.rados.0.smithi138.stdout:1082: write oid 1 current snap is 122 2023-06-27T12:42:35.468 INFO:tasks.rados.rados.0.smithi138.stdout:1082: seq_num 528 ranges {474096=559829,1780963=408850,2800046=74050} 2023-06-27T12:42:35.473 INFO:tasks.rados.rados.0.smithi138.stdout:1082: writing smithi138138488-1 from 474096 to 1033925 tid 1 2023-06-27T12:42:35.482 INFO:tasks.rados.rados.0.smithi138.stdout:1082: writing smithi138138488-1 from 1780963 to 2189813 tid 2 2023-06-27T12:42:35.486 INFO:tasks.rados.rados.0.smithi138.stdout:1082: writing smithi138138488-1 from 2800046 to 2874096 tid 3 2023-06-27T12:42:35.486 INFO:tasks.rados.rados.0.smithi138.stdout:1083: snap_remove snap 121 2023-06-27T12:42:35.788 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:35.913 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:35.908+0000 7f7d4fad6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:35.912+0000 7f7d4fad6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:35.962 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:35.965 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:35.965 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:36.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~121s 2023-06-27T12:42:36.437 INFO:tasks.rados.rados.0.smithi138.stdout:1083: done (1 left) 2023-06-27T12:42:36.437 INFO:tasks.rados.rados.0.smithi138.stdout:1084: read oid 7 snap 113 2023-06-27T12:42:36.437 INFO:tasks.rados.rados.0.smithi138.stdout:1084: expect deleted 2023-06-27T12:42:36.438 INFO:tasks.rados.rados.0.smithi138.stdout:1085: write (excl) oid 9 current snap is 122 2023-06-27T12:42:36.438 INFO:tasks.rados.rados.0.smithi138.stdout:1082: finishing write tid 1 to smithi138138488-1 2023-06-27T12:42:36.438 INFO:tasks.rados.rados.0.smithi138.stdout:1085: seq_num 529 ranges {490903=635078,1839445=552508} 2023-06-27T12:42:36.443 INFO:tasks.rados.rados.0.smithi138.stdout:1085: writing smithi138138488-9 from 490903 to 1125981 tid 1 2023-06-27T12:42:36.451 INFO:tasks.rados.rados.0.smithi138.stdout:1085: writing smithi138138488-9 from 1839445 to 2391953 tid 2 2023-06-27T12:42:36.451 INFO:tasks.rados.rados.0.smithi138.stdout:1082: finishing write tid 2 to smithi138138488-1 2023-06-27T12:42:36.451 INFO:tasks.rados.rados.0.smithi138.stdout:1082: finishing write tid 3 to smithi138138488-1 2023-06-27T12:42:36.451 INFO:tasks.rados.rados.0.smithi138.stdout:1082: finishing write tid 4 to smithi138138488-1 2023-06-27T12:42:36.452 INFO:tasks.rados.rados.0.smithi138.stdout:1086: read oid 47 snap -1 2023-06-27T12:42:36.452 INFO:tasks.rados.rados.0.smithi138.stdout:1086: expect (ObjNum 463 snap 108 seq_num 463) 2023-06-27T12:42:36.452 INFO:tasks.rados.rados.0.smithi138.stdout:1087: read oid 13 snap -1 2023-06-27T12:42:36.452 INFO:tasks.rados.rados.0.smithi138.stdout:1087: expect deleted 2023-06-27T12:42:36.452 INFO:tasks.rados.rados.0.smithi138.stdout:1088: read oid 20 snap -1 2023-06-27T12:42:36.452 INFO:tasks.rados.rados.0.smithi138.stdout:1088: expect (ObjNum 479 snap 112 seq_num 479) 2023-06-27T12:42:36.452 INFO:tasks.rados.rados.0.smithi138.stdout:1089: read oid 14 snap -1 2023-06-27T12:42:36.452 INFO:tasks.rados.rados.0.smithi138.stdout:1089: expect (ObjNum 499 snap 113 seq_num 499) 2023-06-27T12:42:36.453 INFO:tasks.rados.rados.0.smithi138.stdout:1090: write oid 8 current snap is 122 2023-06-27T12:42:36.453 INFO:tasks.rados.rados.0.smithi138.stdout:1082: finishing write tid 5 to smithi138138488-1 2023-06-27T12:42:36.453 INFO:tasks.rados.rados.0.smithi138.stdout:1082: oid 1 updating version 0 to 307 2023-06-27T12:42:36.453 INFO:tasks.rados.rados.0.smithi138.stdout:1082: oid 1 version 307 is already newer than 307 2023-06-27T12:42:36.453 INFO:tasks.rados.rados.0.smithi138.stdout:1082: oid 1 version 307 is already newer than 307 2023-06-27T12:42:36.453 INFO:tasks.rados.rados.0.smithi138.stdout:1082: oid 1 version 307 is already newer than 307 2023-06-27T12:42:36.453 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 307 (ObjNum 528 snap 122 seq_num 528) dirty exists 2023-06-27T12:42:36.453 INFO:tasks.rados.rados.0.smithi138.stdout:1082: left oid 1 (ObjNum 528 snap 122 seq_num 528) 2023-06-27T12:42:36.454 INFO:tasks.rados.rados.0.smithi138.stdout:1090: seq_num 530 ranges {507710=710327,1897927=412519,2854892=52818} 2023-06-27T12:42:36.457 INFO:tasks.rados.rados.0.smithi138.stdout:1090: writing smithi138138488-8 from 507710 to 1218037 tid 1 2023-06-27T12:42:36.464 INFO:tasks.rados.rados.0.smithi138.stdout:1090: writing smithi138138488-8 from 1897927 to 2310446 tid 2 2023-06-27T12:42:36.467 INFO:tasks.rados.rados.0.smithi138.stdout:1090: writing smithi138138488-8 from 2854892 to 2907710 tid 3 2023-06-27T12:42:36.467 INFO:tasks.rados.rados.0.smithi138.stdout:1082: done (7 left) 2023-06-27T12:42:36.468 INFO:tasks.rados.rados.0.smithi138.stdout:1091: read oid 31 snap -1 2023-06-27T12:42:36.468 INFO:tasks.rados.rados.0.smithi138.stdout:1091: expect (ObjNum 525 snap 121 seq_num 525) 2023-06-27T12:42:36.468 INFO:tasks.rados.rados.0.smithi138.stdout:1092: read oid 26 snap -1 2023-06-27T12:42:36.468 INFO:tasks.rados.rados.0.smithi138.stdout:1092: expect (ObjNum 451 snap 104 seq_num 451) 2023-06-27T12:42:36.468 INFO:tasks.rados.rados.0.smithi138.stdout:1093: snap_remove snap 118 2023-06-27T12:42:37.439 INFO:tasks.rados.rados.0.smithi138.stdout:1093: done (9 left) 2023-06-27T12:42:37.439 INFO:tasks.rados.rados.0.smithi138.stdout:1094: read oid 22 snap -1 2023-06-27T12:42:37.440 INFO:tasks.rados.rados.0.smithi138.stdout:1094: expect (ObjNum 492 snap 113 seq_num 492) 2023-06-27T12:42:37.440 INFO:tasks.rados.rados.0.smithi138.stdout:1095: read oid 37 snap -1 2023-06-27T12:42:37.440 INFO:tasks.rados.rados.0.smithi138.stdout:1095: expect (ObjNum 518 snap 121 seq_num 518) 2023-06-27T12:42:37.441 INFO:tasks.rados.rados.0.smithi138.stdout:1096: write oid 44 current snap is 122 2023-06-27T12:42:37.441 INFO:tasks.rados.rados.0.smithi138.stdout:1096: seq_num 531 ranges {524517=785576,1956409=556177} 2023-06-27T12:42:37.445 INFO:tasks.rados.rados.0.smithi138.stdout:1096: writing smithi138138488-44 from 524517 to 1310093 tid 1 2023-06-27T12:42:37.452 INFO:tasks.rados.rados.0.smithi138.stdout:1096: writing smithi138138488-44 from 1956409 to 2512586 tid 2 2023-06-27T12:42:37.453 INFO:tasks.rados.rados.0.smithi138.stdout:1097: write oid 12 current snap is 122 2023-06-27T12:42:37.453 INFO:tasks.rados.rados.0.smithi138.stdout:1097: seq_num 532 ranges {541324=460825,1498538=416188,2393385=547939} 2023-06-27T12:42:37.458 INFO:tasks.rados.rados.0.smithi138.stdout:1097: writing smithi138138488-12 from 541324 to 1002149 tid 1 2023-06-27T12:42:37.463 INFO:tasks.rados.rados.0.smithi138.stdout:1097: writing smithi138138488-12 from 1498538 to 1914726 tid 2 2023-06-27T12:42:37.469 INFO:tasks.rados.rados.0.smithi138.stdout:1097: writing smithi138138488-12 from 2393385 to 2941324 tid 3 2023-06-27T12:42:37.469 INFO:tasks.rados.rados.0.smithi138.stdout:1098: write (excl) oid 42 current snap is 122 2023-06-27T12:42:37.469 INFO:tasks.rados.rados.0.smithi138.stdout:1098: seq_num 533 ranges {558131=652427,1673373=559846,2820808=137323} 2023-06-27T12:42:37.476 INFO:tasks.rados.rados.0.smithi138.stdout:1098: writing smithi138138488-42 from 558131 to 1210558 tid 1 2023-06-27T12:42:37.482 INFO:tasks.rados.rados.0.smithi138.stdout:1098: writing smithi138138488-42 from 1673373 to 2233219 tid 2 2023-06-27T12:42:37.486 INFO:tasks.rados.rados.0.smithi138.stdout:1098: writing smithi138138488-42 from 2820808 to 2958131 tid 3 2023-06-27T12:42:37.486 INFO:tasks.rados.rados.0.smithi138.stdout:1099: snap_create 2023-06-27T12:42:37.486 INFO:tasks.rados.rados.0.smithi138.stdout:1086: expect (ObjNum 463 snap 108 seq_num 463) 2023-06-27T12:42:37.693 INFO:tasks.rados.rados.0.smithi138.stdout:1088: expect (ObjNum 479 snap 112 seq_num 479) 2023-06-27T12:42:37.697 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:42:37.698 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:42:37.698 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:37.793 INFO:tasks.rados.rados.0.smithi138.stdout:1085: finishing write tid 1 to smithi138138488-9 2023-06-27T12:42:37.794 INFO:tasks.rados.rados.0.smithi138.stdout:1090: finishing write tid 1 to smithi138138488-8 2023-06-27T12:42:37.794 INFO:tasks.rados.rados.0.smithi138.stdout:1091: expect (ObjNum 525 snap 121 seq_num 525) 2023-06-27T12:42:37.827 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:37.820+0000 7fd94a019700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:37.830 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:37.826+0000 7fd94a019700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:37.903 INFO:tasks.rados.rados.0.smithi138.stdout:1090: finishing write tid 2 to smithi138138488-8 2023-06-27T12:42:37.904 INFO:tasks.rados.rados.0.smithi138.stdout:1092: expect (ObjNum 451 snap 104 seq_num 451) 2023-06-27T12:42:37.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:38.037 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:38.039 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:38.051 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2023-06-27T12:42:37.966433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":84676273,"num_objects":257,"num_object_clones":211,"num_object_copies":771,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1567,"num_read_kb":1118335,"num_write":1365,"num_write_kb":396474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2570,"ondisk_log_size":2570,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":799680,"kb_used_data":671904,"kb_used_omap":0,"kb_used_meta":127744,"kb_avail":376687680,"statfs":{"total":386547056640,"available":385728184320,"internally_reserved":0,"allocated":688029696,"data_stored":776543604,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":130809856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4353112,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":41141,"num_write":28,"num_write_kb":8285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000931"},"pg_stats":[{"pgid":"3.f","version":"256'32","reported_seq":617,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.585546+0000","last_change":"2023-06-27T12:42:33.588847+0000","last_active":"2023-06-27T12:42:33.585546+0000","last_peered":"2023-06-27T12:42:33.585546+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:33.585546+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2689219,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":10,"num_read_kb":5,"num_write":23,"num_write_kb":6052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"254'159","reported_seq":813,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.589325+0000","last_change":"2023-06-27T12:42:33.600765+0000","last_active":"2023-06-27T12:42:33.589325+0000","last_peered":"2023-06-27T12:42:33.589325+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:33.589325+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12000528,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":135172,"num_write":114,"num_write_kb":33537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"260'197","reported_seq":813,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.932186+0000","last_change":"2023-06-27T12:42:33.689080+0000","last_active":"2023-06-27T12:42:33.932186+0000","last_peered":"2023-06-27T12:42:33.932186+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:33.932186+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14017434,"num_objects":24,"num_object_clones":19,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":77565,"num_write":145,"num_write_kb":41867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"230'100","reported_seq":746,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.586830+0000","last_change":"2023-06-27T12:42:33.630475+0000","last_active":"2023-06-27T12:42:33.586830+0000","last_peered":"2023-06-27T12:42:33.586830+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:33.586830+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9865907,"num_objects":17,"num_object_clones":13,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":96,"num_read_kb":54526,"num_write":79,"num_write_kb":22429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"254'69","reported_seq":317,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.593568+0000","last_change":"2023-06-27T12:42:33.616259+0000","last_active":"2023-06-27T12:42:33.593568+0000","last_peered":"2023-06-27T12:42:33.593568+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:33.593568+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":68,"num_read_kb":34135,"num_write":29,"num_write_kb":8919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"260'326","reported_seq":834,"reported_epoch":260,"state":"active+clean","last_fresh":"2023-06-27T12:42:33.708561+0000","last_change":"2023-06-27T12:42:33.616803+0000","last_active":"2023-06-27T12:42:33.708561+0000","last_peered":"2023-06-27T12:42:33.708561+0000","last_clean":"2023-06-27T12:42:33.708561+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:33.708561+0000","last_undegraded":"2023-06-27T12:42:33.708561+0000","last_fullsized":"2023-06-27T12:42:33.708561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8672676,"num_objects":37,"num_object_clones":31,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":256,"num_read_kb":160791,"num_write":196,"num_write_kb":56404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"249'154","reported_seq":667,"reported_epoch":260,"state":"active+clean","last_fresh":"2023-06-27T12:42:33.617308+0000","last_change":"2023-06-27T12:42:33.630317+0000","last_active":"2023-06-27T12:42:33.617308+0000","last_peered":"2023-06-27T12:42:33.617308+0000","last_clean":"2023-06-27T12:42:33.617308+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:33.617308+0000","last_undegraded":"2023-06-27T12:42:33.617308+0000","last_fullsized":"2023-06-27T12:42:33.617308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":110,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":673,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.599720+0000","last_change":"2023-06-27T12:42:33.614649+0000","last_active":"2023-06-27T12:42:33.599720+0000","last_peered":"2023-06-27T12:42:33.599720+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:33.599720+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":424,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-06-27T12:42:33.592236+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:33.592236+0000","last_peered":"2023-06-27T12:42:33.592236+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:33.592236+0000","last_undegraded":"2023-06-27T12:42:33.592236+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":587,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-06-27T12:42:33.578363+0000","last_change":"2023-06-27T12:42:33.622951+0000","last_active":"2023-06-27T12:42:33.578363+0000","last_peered":"2023-06-27T12:42:33.578363+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:33.578363+0000","last_undegraded":"2023-06-27T12:42:33.578363+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":407,"reported_epoch":260,"state":"active+clean","last_fresh":"2023-06-27T12:42:33.594610+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:33.594610+0000","last_peered":"2023-06-27T12:42:33.594610+0000","last_clean":"2023-06-27T12:42:33.594610+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:33.594610+0000","last_undegraded":"2023-06-27T12:42:33.594610+0000","last_fullsized":"2023-06-27T12:42:33.594610+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"260'143","reported_seq":756,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.630539+0000","last_change":"2023-06-27T12:42:33.601182+0000","last_active":"2023-06-27T12:42:33.630539+0000","last_peered":"2023-06-27T12:42:33.630539+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:33.630539+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6067492,"num_objects":17,"num_object_clones":12,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":68,"num_read_kb":66414,"num_write":96,"num_write_kb":26954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":406,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-06-27T12:42:33.587977+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:33.587977+0000","last_peered":"2023-06-27T12:42:33.587977+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:33.587977+0000","last_undegraded":"2023-06-27T12:42:33.587977+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"255'331","reported_seq":382,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.594422+0000","last_change":"2023-06-27T12:42:33.603680+0000","last_active":"2023-06-27T12:42:33.594422+0000","last_peered":"2023-06-27T12:42:33.594422+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:33.594422+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621991,"num_objects":11,"num_object_clones":10,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":38,"num_read_kb":24501,"num_write":59,"num_write_kb":18141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":11}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":403,"reported_epoch":260,"state":"active+clean","last_fresh":"2023-06-27T12:42:33.576156+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:33.576156+0000","last_peered":"2023-06-27T12:42:33.576156+0000","last_clean":"2023-06-27T12:42:33.576156+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:33.576156+0000","last_undegraded":"2023-06-27T12:42:33.576156+0000","last_fullsized":"2023-06-27T12:42:33.576156+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":159,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-06-27T12:42:33.611298+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:33.611298+0000","last_peered":"2023-06-27T12:42:33.611298+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:33.611298+0000","last_undegraded":"2023-06-27T12:42:33.611298+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"255'303","reported_seq":942,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.573504+0000","last_change":"2023-06-27T12:42:33.581749+0000","last_active":"2023-06-27T12:42:33.573504+0000","last_peered":"2023-06-27T12:42:33.573504+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:33.573504+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2386693,"num_objects":41,"num_object_clones":35,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":267,"num_read_kb":167573,"num_write":219,"num_write_kb":63612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":41}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":407,"reported_epoch":260,"state":"active+clean","last_fresh":"2023-06-27T12:42:33.603115+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:33.603115+0000","last_peered":"2023-06-27T12:42:33.603115+0000","last_clean":"2023-06-27T12:42:33.603115+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:33.603115+0000","last_undegraded":"2023-06-27T12:42:33.603115+0000","last_fullsized":"2023-06-27T12:42:33.603115+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"260'163","reported_seq":346,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.722030+0000","last_change":"2023-06-27T12:42:33.625783+0000","last_active":"2023-06-27T12:42:33.722030+0000","last_peered":"2023-06-27T12:42:33.722030+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:33.722030+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5311243,"num_objects":9,"num_object_clones":9,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":73,"num_read_kb":41526,"num_write":49,"num_write_kb":13967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-06-27T12:42:33.592353+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:33.592353+0000","last_peered":"2023-06-27T12:42:33.592353+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:33.592353+0000","last_undegraded":"2023-06-27T12:42:33.592353+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"260'148","reported_seq":779,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.783307+0000","last_change":"2023-06-27T12:42:33.596674+0000","last_active":"2023-06-27T12:42:33.783307+0000","last_peered":"2023-06-27T12:42:33.783307+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:33.783307+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462572,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":88,"num_read_kb":99407,"num_write":104,"num_write_kb":31071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":340,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.589346+0000","last_change":"2023-06-27T12:42:33.596461+0000","last_active":"2023-06-27T12:42:33.589346+0000","last_peered":"2023-06-27T12:42:33.589346+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:33.589346+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":42747,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":424,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-06-27T12:42:33.599883+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:33.599883+0000","last_peered":"2023-06-27T12:42:33.599883+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:33.599883+0000","last_undegraded":"2023-06-27T12:42:33.599883+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":260,"state":"active+clean","last_fresh":"2023-06-27T12:42:33.595018+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:33.595018+0000","last_peered":"2023-06-27T12:42:33.595018+0000","last_clean":"2023-06-27T12:42:33.595018+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:33.595018+0000","last_undegraded":"2023-06-27T12:42:33.595018+0000","last_fullsized":"2023-06-27T12:42:33.595018+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"252'177","reported_seq":774,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:33.591097+0000","last_change":"2023-06-27T12:42:33.602788+0000","last_active":"2023-06-27T12:42:33.591097+0000","last_peered":"2023-06-27T12:42:33.591097+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:33.591097+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739772,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":89,"num_read_kb":88491,"num_write":92,"num_write_kb":26834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":84676273,"num_objects":257,"num_object_clones":211,"num_object_copies":771,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1567,"num_read_kb":1118335,"num_write":1365,"num_write_kb":396474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2473,"ondisk_log_size":2473,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673240,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":191284,"kb_used_data":160364,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94180556,"statfs":{"total":96636764160,"available":96440889344,"internally_reserved":0,"allocated":164212736,"data_stored":185491091,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":203400,"kb_used_data":172480,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94168440,"statfs":{"total":96636764160,"available":96428482560,"internally_reserved":0,"allocated":176619520,"data_stored":197676107,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673240,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":223716,"kb_used_data":188700,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94148124,"statfs":{"total":96636764160,"available":96407678976,"internally_reserved":0,"allocated":193228800,"data_stored":216336854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:38.100 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:38.091+0000 7f3e0365a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:38.100 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:38.095+0000 7f3e0365a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:38.109 INFO:tasks.rados.rados.0.smithi138.stdout:1090: finishing write tid 3 to smithi138138488-8 2023-06-27T12:42:38.109 INFO:tasks.rados.rados.0.smithi138.stdout:1090: finishing write tid 4 to smithi138138488-8 2023-06-27T12:42:38.109 INFO:tasks.rados.rados.0.smithi138.stdout:1090: finishing write tid 5 to smithi138138488-8 2023-06-27T12:42:38.109 INFO:tasks.rados.rados.0.smithi138.stdout:1090: oid 8 updating version 0 to 73 2023-06-27T12:42:38.109 INFO:tasks.rados.rados.0.smithi138.stdout:1090: oid 8 version 73 is already newer than 73 2023-06-27T12:42:38.110 INFO:tasks.rados.rados.0.smithi138.stdout:1090: oid 8 version 73 is already newer than 73 2023-06-27T12:42:38.110 INFO:tasks.rados.rados.0.smithi138.stdout:1090: oid 8 version 73 is already newer than 73 2023-06-27T12:42:38.110 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 8 v 73 (ObjNum 530 snap 122 seq_num 530) dirty exists 2023-06-27T12:42:38.110 INFO:tasks.rados.rados.0.smithi138.stdout:1090: left oid 8 (ObjNum 530 snap 122 seq_num 530) 2023-06-27T12:42:38.110 INFO:tasks.rados.rados.0.smithi138.stdout:1085: finishing write tid 2 to smithi138138488-9 2023-06-27T12:42:38.110 INFO:tasks.rados.rados.0.smithi138.stdout:1085: finishing write tid 3 to smithi138138488-9 2023-06-27T12:42:38.110 INFO:tasks.rados.rados.0.smithi138.stdout:1085: finishing write tid 4 to smithi138138488-9 2023-06-27T12:42:38.111 INFO:tasks.rados.rados.0.smithi138.stdout:1085: oid 9 updating version 0 to 308 2023-06-27T12:42:38.111 INFO:tasks.rados.rados.0.smithi138.stdout:1085: oid 9 updating version 308 to 310 2023-06-27T12:42:38.111 INFO:tasks.rados.rados.0.smithi138.stdout:1085: oid 9 version 310 is already newer than 310 2023-06-27T12:42:38.111 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 310 (ObjNum 529 snap 122 seq_num 529) dirty exists 2023-06-27T12:42:38.111 INFO:tasks.rados.rados.0.smithi138.stdout:1085: left oid 9 (ObjNum 529 snap 122 seq_num 529) 2023-06-27T12:42:38.111 INFO:tasks.rados.rados.0.smithi138.stdout:1089: expect (ObjNum 499 snap 113 seq_num 499) 2023-06-27T12:42:38.144 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:38.146 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:38.147 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:38.231 INFO:tasks.rados.rados.0.smithi138.stdout:1094: expect (ObjNum 492 snap 113 seq_num 492) 2023-06-27T12:42:38.348 INFO:tasks.rados.rados.0.smithi138.stdout:1096: finishing write tid 1 to smithi138138488-44 2023-06-27T12:42:38.349 INFO:tasks.rados.rados.0.smithi138.stdout:1098: finishing write tid 1 to smithi138138488-42 2023-06-27T12:42:38.349 INFO:tasks.rados.rados.0.smithi138.stdout:1095: expect (ObjNum 518 snap 121 seq_num 518) 2023-06-27T12:42:38.498 INFO:tasks.rados.rados.0.smithi138.stdout:1096: finishing write tid 2 to smithi138138488-44 2023-06-27T12:42:38.498 INFO:tasks.rados.rados.0.smithi138.stdout:1098: finishing write tid 2 to smithi138138488-42 2023-06-27T12:42:38.498 INFO:tasks.rados.rados.0.smithi138.stdout:1096: finishing write tid 3 to smithi138138488-44 2023-06-27T12:42:38.498 INFO:tasks.rados.rados.0.smithi138.stdout:1096: finishing write tid 4 to smithi138138488-44 2023-06-27T12:42:38.499 INFO:tasks.rados.rados.0.smithi138.stdout:1096: oid 44 updating version 0 to 337 2023-06-27T12:42:38.499 INFO:tasks.rados.rados.0.smithi138.stdout:1096: oid 44 version 337 is already newer than 337 2023-06-27T12:42:38.499 INFO:tasks.rados.rados.0.smithi138.stdout:1096: oid 44 version 337 is already newer than 337 2023-06-27T12:42:38.499 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 337 (ObjNum 531 snap 122 seq_num 531) dirty exists 2023-06-27T12:42:38.499 INFO:tasks.rados.rados.0.smithi138.stdout:1096: left oid 44 (ObjNum 531 snap 122 seq_num 531) 2023-06-27T12:42:38.499 INFO:tasks.rados.rados.0.smithi138.stdout:1098: finishing write tid 3 to smithi138138488-42 2023-06-27T12:42:38.499 INFO:tasks.rados.rados.0.smithi138.stdout:1084: done (14 left) 2023-06-27T12:42:38.500 INFO:tasks.rados.rados.0.smithi138.stdout:1085: done (13 left) 2023-06-27T12:42:38.500 INFO:tasks.rados.rados.0.smithi138.stdout:1086: done (12 left) 2023-06-27T12:42:38.500 INFO:tasks.rados.rados.0.smithi138.stdout:1087: done (11 left) 2023-06-27T12:42:38.500 INFO:tasks.rados.rados.0.smithi138.stdout:1088: done (10 left) 2023-06-27T12:42:38.500 INFO:tasks.rados.rados.0.smithi138.stdout:1089: done (9 left) 2023-06-27T12:42:38.500 INFO:tasks.rados.rados.0.smithi138.stdout:1090: done (8 left) 2023-06-27T12:42:38.500 INFO:tasks.rados.rados.0.smithi138.stdout:1091: done (7 left) 2023-06-27T12:42:38.501 INFO:tasks.rados.rados.0.smithi138.stdout:1092: done (6 left) 2023-06-27T12:42:38.501 INFO:tasks.rados.rados.0.smithi138.stdout:1094: done (5 left) 2023-06-27T12:42:38.501 INFO:tasks.rados.rados.0.smithi138.stdout:1095: done (4 left) 2023-06-27T12:42:38.501 INFO:tasks.rados.rados.0.smithi138.stdout:1096: done (3 left) 2023-06-27T12:42:38.501 INFO:tasks.rados.rados.0.smithi138.stdout:1099: done (2 left) 2023-06-27T12:42:38.501 INFO:tasks.rados.rados.0.smithi138.stdout:1100: write oid 35 current snap is 123 2023-06-27T12:42:38.502 INFO:tasks.rados.rados.0.smithi138.stdout:1098: finishing write tid 4 to smithi138138488-42 2023-06-27T12:42:38.502 INFO:tasks.rados.rados.0.smithi138.stdout:1100: seq_num 534 ranges {574938=727676,1731855=703504,2848231=126707} 2023-06-27T12:42:38.504 INFO:tasks.rados.rados.0.smithi138.stdout:1100: writing smithi138138488-35 from 574938 to 1302614 tid 1 2023-06-27T12:42:38.510 INFO:tasks.rados.rados.0.smithi138.stdout:1100: writing smithi138138488-35 from 1731855 to 2435359 tid 2 2023-06-27T12:42:38.515 INFO:tasks.rados.rados.0.smithi138.stdout:1100: writing smithi138138488-35 from 2848231 to 2974938 tid 3 2023-06-27T12:42:38.515 INFO:tasks.rados.rados.0.smithi138.stdout:1101: read oid 19 snap -1 2023-06-27T12:42:38.515 INFO:tasks.rados.rados.0.smithi138.stdout:1101: expect deleted 2023-06-27T12:42:38.515 INFO:tasks.rados.rados.0.smithi138.stdout:1102: delete oid 3 current snap is 123 2023-06-27T12:42:38.516 INFO:tasks.rados.rados.0.smithi138.stdout:1098: finishing write tid 5 to smithi138138488-42 2023-06-27T12:42:38.516 INFO:tasks.rados.rados.0.smithi138.stdout:1098: oid 42 updating version 0 to 107 2023-06-27T12:42:38.516 INFO:tasks.rados.rados.0.smithi138.stdout:1098: oid 42 version 107 is already newer than 107 2023-06-27T12:42:38.517 INFO:tasks.rados.rados.0.smithi138.stdout:1098: oid 42 version 107 is already newer than 103 2023-06-27T12:42:38.517 INFO:tasks.rados.rados.0.smithi138.stdout:1098: oid 42 version 107 is already newer than 107 2023-06-27T12:42:38.517 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 107 (ObjNum 533 snap 122 seq_num 533) dirty exists 2023-06-27T12:42:38.517 INFO:tasks.rados.rados.0.smithi138.stdout:1098: left oid 42 (ObjNum 533 snap 122 seq_num 533) 2023-06-27T12:42:38.518 INFO:tasks.rados.rados.0.smithi138.stdout:1097: finishing write tid 1 to smithi138138488-12 2023-06-27T12:42:38.518 INFO:tasks.rados.rados.0.smithi138.stdout:1097: finishing write tid 2 to smithi138138488-12 2023-06-27T12:42:38.518 INFO:tasks.rados.rados.0.smithi138.stdout:1097: finishing write tid 3 to smithi138138488-12 2023-06-27T12:42:38.518 INFO:tasks.rados.rados.0.smithi138.stdout:1097: finishing write tid 4 to smithi138138488-12 2023-06-27T12:42:38.519 INFO:tasks.rados.rados.0.smithi138.stdout:1097: finishing write tid 5 to smithi138138488-12 2023-06-27T12:42:38.519 INFO:tasks.rados.rados.0.smithi138.stdout:1097: oid 12 updating version 0 to 209 2023-06-27T12:42:38.519 INFO:tasks.rados.rados.0.smithi138.stdout:1097: oid 12 version 209 is already newer than 209 2023-06-27T12:42:38.519 INFO:tasks.rados.rados.0.smithi138.stdout:1097: oid 12 version 209 is already newer than 209 2023-06-27T12:42:38.519 INFO:tasks.rados.rados.0.smithi138.stdout:1097: oid 12 version 209 is already newer than 209 2023-06-27T12:42:38.519 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 209 (ObjNum 532 snap 122 seq_num 532) dirty exists 2023-06-27T12:42:38.519 INFO:tasks.rados.rados.0.smithi138.stdout:1097: left oid 12 (ObjNum 532 snap 122 seq_num 532) 2023-06-27T12:42:38.563 INFO:tasks.rados.rados.0.smithi138.stdout:1100: finishing write tid 1 to smithi138138488-35 2023-06-27T12:42:38.563 INFO:tasks.rados.rados.0.smithi138.stdout:1100: finishing write tid 2 to smithi138138488-35 2023-06-27T12:42:38.564 INFO:tasks.rados.rados.0.smithi138.stdout:1100: finishing write tid 3 to smithi138138488-35 2023-06-27T12:42:38.566 INFO:tasks.rados.rados.0.smithi138.stdout:1100: finishing write tid 4 to smithi138138488-35 2023-06-27T12:42:38.570 INFO:tasks.rados.rados.0.smithi138.stdout:1100: finishing write tid 5 to smithi138138488-35 2023-06-27T12:42:38.571 INFO:tasks.rados.rados.0.smithi138.stdout:1100: oid 35 updating version 0 to 336 2023-06-27T12:42:38.571 INFO:tasks.rados.rados.0.smithi138.stdout:1100: oid 35 version 336 is already newer than 336 2023-06-27T12:42:38.571 INFO:tasks.rados.rados.0.smithi138.stdout:1100: oid 35 version 336 is already newer than 336 2023-06-27T12:42:38.571 INFO:tasks.rados.rados.0.smithi138.stdout:1100: oid 35 version 336 is already newer than 336 2023-06-27T12:42:38.571 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 336 (ObjNum 534 snap 123 seq_num 534) dirty exists 2023-06-27T12:42:38.571 INFO:tasks.rados.rados.0.smithi138.stdout:1100: left oid 35 (ObjNum 534 snap 123 seq_num 534) 2023-06-27T12:42:38.574 INFO:tasks.rados.rados.0.smithi138.stdout:1097: done (4 left) 2023-06-27T12:42:38.574 INFO:tasks.rados.rados.0.smithi138.stdout:1098: done (3 left) 2023-06-27T12:42:38.574 INFO:tasks.rados.rados.0.smithi138.stdout:1100: done (2 left) 2023-06-27T12:42:38.575 INFO:tasks.rados.rados.0.smithi138.stdout:1101: done (1 left) 2023-06-27T12:42:38.575 INFO:tasks.rados.rados.0.smithi138.stdout:1102: done (0 left) 2023-06-27T12:42:38.575 INFO:tasks.rados.rados.0.smithi138.stdout:1103: snap_remove snap 122 2023-06-27T12:42:39.577 INFO:tasks.rados.rados.0.smithi138.stdout:1103: done (0 left) 2023-06-27T12:42:39.578 INFO:tasks.rados.rados.0.smithi138.stdout:1104: snap_remove snap 116 2023-06-27T12:42:40.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:40.287 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:40.281+0000 7ffb84b40700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:40.289 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:40.286+0000 7ffb84b40700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:40.338 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:40.338 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:40.578 INFO:tasks.rados.rados.0.smithi138.stdout:1104: done (0 left) 2023-06-27T12:42:40.578 INFO:tasks.rados.rados.0.smithi138.stdout:1105: snap_create 2023-06-27T12:42:41.579 INFO:tasks.rados.rados.0.smithi138.stdout:1105: done (0 left) 2023-06-27T12:42:41.580 INFO:tasks.rados.rados.0.smithi138.stdout:1106: write (excl) oid 40 current snap is 124 2023-06-27T12:42:41.580 INFO:tasks.rados.rados.0.smithi138.stdout:1106: seq_num 536 ranges {608552=478174,1848819=707173} 2023-06-27T12:42:41.587 INFO:tasks.rados.rados.0.smithi138.stdout:1106: writing smithi138138488-40 from 608552 to 1086726 tid 1 2023-06-27T12:42:41.598 INFO:tasks.rados.rados.0.smithi138.stdout:1106: writing smithi138138488-40 from 1848819 to 2555992 tid 2 2023-06-27T12:42:41.599 INFO:tasks.rados.rados.0.smithi138.stdout:1107: write (excl) oid 48 current snap is 124 2023-06-27T12:42:41.599 INFO:tasks.rados.rados.0.smithi138.stdout:1107: seq_num 537 ranges {625359=553423,1790948=567184,2930500=94859} 2023-06-27T12:42:41.607 INFO:tasks.rados.rados.0.smithi138.stdout:1107: writing smithi138138488-48 from 625359 to 1178782 tid 1 2023-06-27T12:42:41.615 INFO:tasks.rados.rados.0.smithi138.stdout:1107: writing smithi138138488-48 from 1790948 to 2358132 tid 2 2023-06-27T12:42:41.620 INFO:tasks.rados.rados.0.smithi138.stdout:1107: writing smithi138138488-48 from 2930500 to 3025359 tid 3 2023-06-27T12:42:41.620 INFO:tasks.rados.rados.0.smithi138.stdout:1108: write (excl) oid 13 current snap is 124 2023-06-27T12:42:41.620 INFO:tasks.rados.rados.0.smithi138.stdout:1108: seq_num 538 ranges {642166=628672,1849430=710842} 2023-06-27T12:42:41.629 INFO:tasks.rados.rados.0.smithi138.stdout:1108: writing smithi138138488-13 from 642166 to 1270838 tid 1 2023-06-27T12:42:41.638 INFO:tasks.rados.rados.0.smithi138.stdout:1108: writing smithi138138488-13 from 1849430 to 2560272 tid 2 2023-06-27T12:42:41.639 INFO:tasks.rados.rados.0.smithi138.stdout:1109: write (excl) oid 43 current snap is 124 2023-06-27T12:42:41.639 INFO:tasks.rados.rados.0.smithi138.stdout:1109: seq_num 539 ranges {658973=703921,1907912=570853,2985346=73627} 2023-06-27T12:42:41.648 INFO:tasks.rados.rados.0.smithi138.stdout:1109: writing smithi138138488-43 from 658973 to 1362894 tid 1 2023-06-27T12:42:41.658 INFO:tasks.rados.rados.0.smithi138.stdout:1109: writing smithi138138488-43 from 1907912 to 2478765 tid 2 2023-06-27T12:42:41.661 INFO:tasks.rados.rados.0.smithi138.stdout:1109: writing smithi138138488-43 from 2985346 to 3058973 tid 3 2023-06-27T12:42:41.662 INFO:tasks.rados.rados.0.smithi138.stdout:1110: snap_remove snap 120 2023-06-27T12:42:41.903 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~127s 2023-06-27T12:42:42.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:42.466+0000 7f23c61b5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:42.474 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:42.470+0000 7f23c61b5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:42.519 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:42.522 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:42.522 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:42.595 INFO:tasks.rados.rados.0.smithi138.stdout:1110: done (4 left) 2023-06-27T12:42:42.595 INFO:tasks.rados.rados.0.smithi138.stdout:1111: write oid 3 current snap is 124 2023-06-27T12:42:42.595 INFO:tasks.rados.rados.0.smithi138.stdout:1107: finishing write tid 1 to smithi138138488-48 2023-06-27T12:42:42.595 INFO:tasks.rados.rados.0.smithi138.stdout:1106: finishing write tid 1 to smithi138138488-40 2023-06-27T12:42:42.595 INFO:tasks.rados.rados.0.smithi138.stdout:1107: finishing write tid 2 to smithi138138488-48 2023-06-27T12:42:42.596 INFO:tasks.rados.rados.0.smithi138.stdout:1106: finishing write tid 2 to smithi138138488-40 2023-06-27T12:42:42.596 INFO:tasks.rados.rados.0.smithi138.stdout:1106: finishing write tid 3 to smithi138138488-40 2023-06-27T12:42:42.596 INFO:tasks.rados.rados.0.smithi138.stdout:1106: finishing write tid 4 to smithi138138488-40 2023-06-27T12:42:42.596 INFO:tasks.rados.rados.0.smithi138.stdout:1106: oid 40 updating version 0 to 111 2023-06-27T12:42:42.596 INFO:tasks.rados.rados.0.smithi138.stdout:1106: oid 40 version 111 is already newer than 111 2023-06-27T12:42:42.596 INFO:tasks.rados.rados.0.smithi138.stdout:1106: oid 40 version 111 is already newer than 108 2023-06-27T12:42:42.596 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 111 (ObjNum 536 snap 124 seq_num 536) dirty exists 2023-06-27T12:42:42.597 INFO:tasks.rados.rados.0.smithi138.stdout:1106: left oid 40 (ObjNum 536 snap 124 seq_num 536) 2023-06-27T12:42:42.597 INFO:tasks.rados.rados.0.smithi138.stdout:1108: finishing write tid 1 to smithi138138488-13 2023-06-27T12:42:42.597 INFO:tasks.rados.rados.0.smithi138.stdout:1107: finishing write tid 3 to smithi138138488-48 2023-06-27T12:42:42.597 INFO:tasks.rados.rados.0.smithi138.stdout:1107: finishing write tid 4 to smithi138138488-48 2023-06-27T12:42:42.597 INFO:tasks.rados.rados.0.smithi138.stdout:1111: seq_num 540 ranges {675780=495523,1682747=714511,3012769=63011} 2023-06-27T12:42:42.603 INFO:tasks.rados.rados.0.smithi138.stdout:1111: writing smithi138138488-3 from 675780 to 1171303 tid 1 2023-06-27T12:42:42.613 INFO:tasks.rados.rados.0.smithi138.stdout:1111: writing smithi138138488-3 from 1682747 to 2397258 tid 2 2023-06-27T12:42:42.617 INFO:tasks.rados.rados.0.smithi138.stdout:1111: writing smithi138138488-3 from 3012769 to 3075780 tid 3 2023-06-27T12:42:42.618 INFO:tasks.rados.rados.0.smithi138.stdout:1106: done (4 left) 2023-06-27T12:42:42.618 INFO:tasks.rados.rados.0.smithi138.stdout:1112: snap_create 2023-06-27T12:42:42.618 INFO:tasks.rados.rados.0.smithi138.stdout:1107: finishing write tid 5 to smithi138138488-48 2023-06-27T12:42:42.618 INFO:tasks.rados.rados.0.smithi138.stdout:1107: oid 48 updating version 0 to 182 2023-06-27T12:42:42.618 INFO:tasks.rados.rados.0.smithi138.stdout:1107: oid 48 updating version 182 to 186 2023-06-27T12:42:42.618 INFO:tasks.rados.rados.0.smithi138.stdout:1107: oid 48 version 186 is already newer than 186 2023-06-27T12:42:42.618 INFO:tasks.rados.rados.0.smithi138.stdout:1107: oid 48 version 186 is already newer than 186 2023-06-27T12:42:42.619 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 186 (ObjNum 537 snap 124 seq_num 537) dirty exists 2023-06-27T12:42:42.619 INFO:tasks.rados.rados.0.smithi138.stdout:1107: left oid 48 (ObjNum 537 snap 124 seq_num 537) 2023-06-27T12:42:42.619 INFO:tasks.rados.rados.0.smithi138.stdout:1109: finishing write tid 1 to smithi138138488-43 2023-06-27T12:42:42.619 INFO:tasks.rados.rados.0.smithi138.stdout:1108: finishing write tid 2 to smithi138138488-13 2023-06-27T12:42:42.619 INFO:tasks.rados.rados.0.smithi138.stdout:1108: finishing write tid 3 to smithi138138488-13 2023-06-27T12:42:42.619 INFO:tasks.rados.rados.0.smithi138.stdout:1108: finishing write tid 4 to smithi138138488-13 2023-06-27T12:42:42.619 INFO:tasks.rados.rados.0.smithi138.stdout:1108: oid 13 updating version 0 to 341 2023-06-27T12:42:42.619 INFO:tasks.rados.rados.0.smithi138.stdout:1108: oid 13 updating version 341 to 343 2023-06-27T12:42:42.620 INFO:tasks.rados.rados.0.smithi138.stdout:1108: oid 13 version 343 is already newer than 343 2023-06-27T12:42:42.620 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 343 (ObjNum 538 snap 124 seq_num 538) dirty exists 2023-06-27T12:42:42.620 INFO:tasks.rados.rados.0.smithi138.stdout:1108: left oid 13 (ObjNum 538 snap 124 seq_num 538) 2023-06-27T12:42:42.620 INFO:tasks.rados.rados.0.smithi138.stdout:1109: finishing write tid 2 to smithi138138488-43 2023-06-27T12:42:42.620 INFO:tasks.rados.rados.0.smithi138.stdout:1109: finishing write tid 3 to smithi138138488-43 2023-06-27T12:42:42.620 INFO:tasks.rados.rados.0.smithi138.stdout:1109: finishing write tid 4 to smithi138138488-43 2023-06-27T12:42:42.620 INFO:tasks.rados.rados.0.smithi138.stdout:1109: finishing write tid 5 to smithi138138488-43 2023-06-27T12:42:42.620 INFO:tasks.rados.rados.0.smithi138.stdout:1109: oid 43 updating version 0 to 163 2023-06-27T12:42:42.621 INFO:tasks.rados.rados.0.smithi138.stdout:1109: oid 43 version 163 is already newer than 159 2023-06-27T12:42:42.621 INFO:tasks.rados.rados.0.smithi138.stdout:1109: oid 43 version 163 is already newer than 163 2023-06-27T12:42:42.621 INFO:tasks.rados.rados.0.smithi138.stdout:1109: oid 43 version 163 is already newer than 163 2023-06-27T12:42:42.621 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 163 (ObjNum 539 snap 124 seq_num 539) dirty exists 2023-06-27T12:42:42.621 INFO:tasks.rados.rados.0.smithi138.stdout:1109: left oid 43 (ObjNum 539 snap 124 seq_num 539) 2023-06-27T12:42:42.670 INFO:tasks.rados.rados.0.smithi138.stdout:1111: finishing write tid 1 to smithi138138488-3 2023-06-27T12:42:42.671 INFO:tasks.rados.rados.0.smithi138.stdout:1111: finishing write tid 2 to smithi138138488-3 2023-06-27T12:42:42.674 INFO:tasks.rados.rados.0.smithi138.stdout:1111: finishing write tid 3 to smithi138138488-3 2023-06-27T12:42:42.676 INFO:tasks.rados.rados.0.smithi138.stdout:1111: finishing write tid 4 to smithi138138488-3 2023-06-27T12:42:42.677 INFO:tasks.rados.rados.0.smithi138.stdout:1111: finishing write tid 5 to smithi138138488-3 2023-06-27T12:42:42.678 INFO:tasks.rados.rados.0.smithi138.stdout:1111: oid 3 updating version 0 to 169 2023-06-27T12:42:42.678 INFO:tasks.rados.rados.0.smithi138.stdout:1111: oid 3 version 169 is already newer than 169 2023-06-27T12:42:42.678 INFO:tasks.rados.rados.0.smithi138.stdout:1111: oid 3 version 169 is already newer than 169 2023-06-27T12:42:42.678 INFO:tasks.rados.rados.0.smithi138.stdout:1111: oid 3 version 169 is already newer than 169 2023-06-27T12:42:42.678 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 169 (ObjNum 540 snap 124 seq_num 540) dirty exists 2023-06-27T12:42:42.678 INFO:tasks.rados.rados.0.smithi138.stdout:1111: left oid 3 (ObjNum 540 snap 124 seq_num 540) 2023-06-27T12:42:43.054 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:42:43.055 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:42:43.055 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:43.185 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:43.178+0000 7fbdfc7ca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:43.188 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:43.184+0000 7fbdfc7ca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:43.382 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:43.384 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:43.396 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2023-06-27T12:42:42.589981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":91080070,"num_objects":263,"num_object_clones":217,"num_object_copies":789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1618,"num_read_kb":1167220,"num_write":1408,"num_write_kb":407744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2629,"ondisk_log_size":2629,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":829752,"kb_used_data":697168,"kb_used_omap":0,"kb_used_meta":131840,"kb_avail":376657608,"statfs":{"total":386547056640,"available":385697390592,"internally_reserved":0,"allocated":713900032,"data_stored":803513452,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":135004160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6403797,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":51,"num_read_kb":48885,"num_write":43,"num_write_kb":11270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.154197"},"pg_stats":[{"pgid":"3.f","version":"256'32","reported_seq":630,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.452406+0000","last_change":"2023-06-27T12:42:37.474990+0000","last_active":"2023-06-27T12:42:38.452406+0000","last_peered":"2023-06-27T12:42:38.452406+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:38.452406+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2689219,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":10,"num_read_kb":5,"num_write":23,"num_write_kb":6052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"262'161","reported_seq":834,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.524436+0000","last_change":"2023-06-27T12:42:37.470436+0000","last_active":"2023-06-27T12:42:38.524436+0000","last_peered":"2023-06-27T12:42:38.524436+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:38.524436+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9428958,"num_objects":21,"num_object_clones":17,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":122,"num_read_kb":147869,"num_write":114,"num_write_kb":33537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"264'209","reported_seq":837,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.518469+0000","last_change":"2023-06-27T12:42:37.691235+0000","last_active":"2023-06-27T12:42:38.518469+0000","last_peered":"2023-06-27T12:42:38.518469+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:38.518469+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14168697,"num_objects":24,"num_object_clones":19,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":92611,"num_write":150,"num_write_kb":43261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"264'107","reported_seq":764,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.443209+0000","last_change":"2023-06-27T12:42:37.503496+0000","last_active":"2023-06-27T12:42:38.443209+0000","last_peered":"2023-06-27T12:42:38.443209+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:38.443209+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9647449,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":98,"num_read_kb":54527,"num_write":84,"num_write_kb":23749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":18}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"263'73","reported_seq":335,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.444157+0000","last_change":"2023-06-27T12:42:37.479269+0000","last_active":"2023-06-27T12:42:38.444157+0000","last_peered":"2023-06-27T12:42:38.444157+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:38.444157+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2907710,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":69,"num_read_kb":34136,"num_write":34,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"264'337","reported_seq":859,"reported_epoch":265,"state":"active+clean","last_fresh":"2023-06-27T12:42:38.445250+0000","last_change":"2023-06-27T12:42:37.618147+0000","last_active":"2023-06-27T12:42:38.445250+0000","last_peered":"2023-06-27T12:42:38.445250+0000","last_clean":"2023-06-27T12:42:38.445250+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:38.445250+0000","last_undegraded":"2023-06-27T12:42:38.445250+0000","last_fullsized":"2023-06-27T12:42:38.445250+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14420868,"num_objects":37,"num_object_clones":31,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":266,"num_read_kb":166257,"num_write":205,"num_write_kb":59070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"249'154","reported_seq":680,"reported_epoch":265,"state":"active+clean","last_fresh":"2023-06-27T12:42:38.447472+0000","last_change":"2023-06-27T12:42:37.504226+0000","last_active":"2023-06-27T12:42:38.447472+0000","last_peered":"2023-06-27T12:42:38.447472+0000","last_clean":"2023-06-27T12:42:38.447472+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:38.447472+0000","last_undegraded":"2023-06-27T12:42:38.447472+0000","last_fullsized":"2023-06-27T12:42:38.447472+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":110,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":686,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.518782+0000","last_change":"2023-06-27T12:42:37.469554+0000","last_active":"2023-06-27T12:42:38.518782+0000","last_peered":"2023-06-27T12:42:38.518782+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:38.518782+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":434,"reported_epoch":265,"state":"active+undersized","last_fresh":"2023-06-27T12:42:38.517493+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:38.517493+0000","last_peered":"2023-06-27T12:42:38.517493+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:38.517493+0000","last_undegraded":"2023-06-27T12:42:38.517493+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":600,"reported_epoch":265,"state":"active+undersized","last_fresh":"2023-06-27T12:42:38.523753+0000","last_change":"2023-06-27T12:42:37.495868+0000","last_active":"2023-06-27T12:42:38.523753+0000","last_peered":"2023-06-27T12:42:38.523753+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:38.523753+0000","last_undegraded":"2023-06-27T12:42:38.523753+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":417,"reported_epoch":265,"state":"active+clean","last_fresh":"2023-06-27T12:42:38.454245+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:38.454245+0000","last_peered":"2023-06-27T12:42:38.454245+0000","last_clean":"2023-06-27T12:42:38.454245+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:38.454245+0000","last_undegraded":"2023-06-27T12:42:38.454245+0000","last_fullsized":"2023-06-27T12:42:38.454245+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"261'150","reported_seq":774,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.520892+0000","last_change":"2023-06-27T12:42:37.532209+0000","last_active":"2023-06-27T12:42:38.520892+0000","last_peered":"2023-06-27T12:42:38.520892+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:38.520892+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563315,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":66415,"num_write":101,"num_write_kb":27991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":416,"reported_epoch":265,"state":"active+undersized","last_fresh":"2023-06-27T12:42:38.525574+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:38.525574+0000","last_peered":"2023-06-27T12:42:38.525574+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:38.525574+0000","last_undegraded":"2023-06-27T12:42:38.525574+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"265'336","reported_seq":400,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.555925+0000","last_change":"2023-06-27T12:42:37.458160+0000","last_active":"2023-06-27T12:42:38.555925+0000","last_peered":"2023-06-27T12:42:38.555925+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:38.555925+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3445567,"num_objects":12,"num_object_clones":11,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":39,"num_read_kb":24502,"num_write":64,"num_write_kb":19664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":413,"reported_epoch":265,"state":"active+clean","last_fresh":"2023-06-27T12:42:38.524256+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:38.524256+0000","last_peered":"2023-06-27T12:42:38.524256+0000","last_clean":"2023-06-27T12:42:38.524256+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:38.524256+0000","last_undegraded":"2023-06-27T12:42:38.524256+0000","last_fullsized":"2023-06-27T12:42:38.524256+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":169,"reported_epoch":265,"state":"active+undersized","last_fresh":"2023-06-27T12:42:38.449576+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:38.449576+0000","last_peered":"2023-06-27T12:42:38.449576+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:38.449576+0000","last_undegraded":"2023-06-27T12:42:38.449576+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"263'310","reported_seq":967,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.520670+0000","last_change":"2023-06-27T12:42:37.484074+0000","last_active":"2023-06-27T12:42:38.520670+0000","last_peered":"2023-06-27T12:42:38.520670+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:38.520670+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8151692,"num_objects":41,"num_object_clones":35,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":277,"num_read_kb":173895,"num_write":228,"num_write_kb":65793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":41}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":417,"reported_epoch":265,"state":"active+clean","last_fresh":"2023-06-27T12:42:38.447770+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:38.447770+0000","last_peered":"2023-06-27T12:42:38.447770+0000","last_clean":"2023-06-27T12:42:38.447770+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:38.447770+0000","last_undegraded":"2023-06-27T12:42:38.447770+0000","last_fullsized":"2023-06-27T12:42:38.447770+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"261'167","reported_seq":360,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.521966+0000","last_change":"2023-06-27T12:42:37.603723+0000","last_active":"2023-06-27T12:42:38.521966+0000","last_peered":"2023-06-27T12:42:38.521966+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:38.521966+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722866,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":74,"num_read_kb":41526,"num_write":49,"num_write_kb":13967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":265,"state":"active+undersized","last_fresh":"2023-06-27T12:42:38.518264+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:38.518264+0000","last_peered":"2023-06-27T12:42:38.518264+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:38.518264+0000","last_undegraded":"2023-06-27T12:42:38.518264+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"260'148","reported_seq":795,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.521607+0000","last_change":"2023-06-27T12:42:37.519288+0000","last_active":"2023-06-27T12:42:38.521607+0000","last_peered":"2023-06-27T12:42:38.521607+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:38.521607+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6353211,"num_objects":21,"num_object_clones":17,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":90,"num_read_kb":105198,"num_write":104,"num_write_kb":31071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":353,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.521796+0000","last_change":"2023-06-27T12:42:37.484752+0000","last_active":"2023-06-27T12:42:38.521796+0000","last_peered":"2023-06-27T12:42:38.521796+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:38.521796+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":42747,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":434,"reported_epoch":265,"state":"active+undersized","last_fresh":"2023-06-27T12:42:38.522845+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:38.522845+0000","last_peered":"2023-06-27T12:42:38.522845+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:38.522845+0000","last_undegraded":"2023-06-27T12:42:38.522845+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":413,"reported_epoch":265,"state":"active+clean","last_fresh":"2023-06-27T12:42:38.522292+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:38.522292+0000","last_peered":"2023-06-27T12:42:38.522292+0000","last_clean":"2023-06-27T12:42:38.522292+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:38.522292+0000","last_undegraded":"2023-06-27T12:42:38.522292+0000","last_fullsized":"2023-06-27T12:42:38.522292+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"252'177","reported_seq":789,"reported_epoch":265,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:38.520581+0000","last_change":"2023-06-27T12:42:37.454366+0000","last_active":"2023-06-27T12:42:38.520581+0000","last_peered":"2023-06-27T12:42:38.520581+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:38.520581+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4739772,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":91,"num_read_kb":92050,"num_write":92,"num_write_kb":26834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":91080070,"num_objects":263,"num_object_clones":217,"num_object_copies":789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1618,"num_read_kb":1167220,"num_write":1408,"num_write_kb":407744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2532,"ondisk_log_size":2532,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":199064,"kb_used_data":168144,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94172776,"statfs":{"total":96636764160,"available":96432922624,"internally_reserved":0,"allocated":172179456,"data_stored":194410035,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":217864,"kb_used_data":182492,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94153976,"statfs":{"total":96636764160,"available":96413671424,"internally_reserved":0,"allocated":186871808,"data_stored":208193458,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":231544,"kb_used_data":196172,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94140296,"statfs":{"total":96636764160,"available":96399663104,"internally_reserved":0,"allocated":200880128,"data_stored":223870407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:43.398 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.d 3.b 3.3 3.0 3.4 3.7 3.6 2023-06-27T12:42:43.533 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:43.517+0000 7f89b0b78700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:43.536 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:43.532+0000 7f89b0b78700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:43.598 INFO:tasks.rados.rados.0.smithi138.stdout:1107: done (4 left) 2023-06-27T12:42:43.598 INFO:tasks.rados.rados.0.smithi138.stdout:1108: done (3 left) 2023-06-27T12:42:43.598 INFO:tasks.rados.rados.0.smithi138.stdout:1109: done (2 left) 2023-06-27T12:42:43.598 INFO:tasks.rados.rados.0.smithi138.stdout:1111: done (1 left) 2023-06-27T12:42:43.599 INFO:tasks.rados.rados.0.smithi138.stdout:1112: done (0 left) 2023-06-27T12:42:43.599 INFO:tasks.rados.rados.0.smithi138.stdout:1113: snap_create 2023-06-27T12:42:43.732 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.f] on osd.0 to force-recovery; instructing pg(s) [3.7,3.d] on osd.1 to force-recovery; instructing pg(s) [3.0,3.3,3.4,3.6] on osd.2 to force-recovery; 2023-06-27T12:42:43.733 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:42:44.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:44.603 INFO:tasks.rados.rados.0.smithi138.stdout:1113: done (0 left) 2023-06-27T12:42:44.603 INFO:tasks.rados.rados.0.smithi138.stdout:1114: read oid 17 snap -1 2023-06-27T12:42:44.603 INFO:tasks.rados.rados.0.smithi138.stdout:1114: expect deleted 2023-06-27T12:42:44.604 INFO:tasks.rados.rados.0.smithi138.stdout:1115: write oid 12 current snap is 126 2023-06-27T12:42:44.604 INFO:tasks.rados.rados.0.smithi138.stdout:1115: seq_num 541 ranges {692587=570772,2024876=574522,3040192=52395} 2023-06-27T12:42:44.613 INFO:tasks.rados.rados.0.smithi138.stdout:1115: writing smithi138138488-12 from 692587 to 1263359 tid 1 2023-06-27T12:42:44.623 INFO:tasks.rados.rados.0.smithi138.stdout:1115: writing smithi138138488-12 from 2024876 to 2599398 tid 2 2023-06-27T12:42:44.627 INFO:tasks.rados.rados.0.smithi138.stdout:1115: writing smithi138138488-12 from 3040192 to 3092587 tid 3 2023-06-27T12:42:44.627 INFO:tasks.rados.rados.0.smithi138.stdout:1116: read oid 7 snap -1 2023-06-27T12:42:44.627 INFO:tasks.rados.rados.0.smithi138.stdout:1116: expect deleted 2023-06-27T12:42:44.627 INFO:tasks.rados.rados.0.smithi138.stdout:1117: snap_remove snap 124 2023-06-27T12:42:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:44.667+0000 7f317e235700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:44.672+0000 7f317e235700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:44.788 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:44.791 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:44.792 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:45.434 INFO:tasks.rados.rados.0.smithi138.stdout:1117: done (3 left) 2023-06-27T12:42:45.434 INFO:tasks.rados.rados.0.smithi138.stdout:1118: snap_create 2023-06-27T12:42:45.435 INFO:tasks.rados.rados.0.smithi138.stdout:1115: finishing write tid 1 to smithi138138488-12 2023-06-27T12:42:45.435 INFO:tasks.rados.rados.0.smithi138.stdout:1115: finishing write tid 2 to smithi138138488-12 2023-06-27T12:42:45.435 INFO:tasks.rados.rados.0.smithi138.stdout:1115: finishing write tid 3 to smithi138138488-12 2023-06-27T12:42:45.435 INFO:tasks.rados.rados.0.smithi138.stdout:1115: finishing write tid 4 to smithi138138488-12 2023-06-27T12:42:45.435 INFO:tasks.rados.rados.0.smithi138.stdout:1115: finishing write tid 5 to smithi138138488-12 2023-06-27T12:42:45.435 INFO:tasks.rados.rados.0.smithi138.stdout:1115: oid 12 updating version 0 to 222 2023-06-27T12:42:45.436 INFO:tasks.rados.rados.0.smithi138.stdout:1115: oid 12 version 222 is already newer than 222 2023-06-27T12:42:45.436 INFO:tasks.rados.rados.0.smithi138.stdout:1115: oid 12 version 222 is already newer than 222 2023-06-27T12:42:45.436 INFO:tasks.rados.rados.0.smithi138.stdout:1115: oid 12 version 222 is already newer than 222 2023-06-27T12:42:45.436 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 222 (ObjNum 541 snap 126 seq_num 541) dirty exists 2023-06-27T12:42:45.436 INFO:tasks.rados.rados.0.smithi138.stdout:1115: left oid 12 (ObjNum 541 snap 126 seq_num 541) 2023-06-27T12:42:46.438 INFO:tasks.rados.rados.0.smithi138.stdout:1114: done (3 left) 2023-06-27T12:42:46.439 INFO:tasks.rados.rados.0.smithi138.stdout:1115: done (2 left) 2023-06-27T12:42:46.439 INFO:tasks.rados.rados.0.smithi138.stdout:1116: done (1 left) 2023-06-27T12:42:46.439 INFO:tasks.rados.rados.0.smithi138.stdout:1118: done (0 left) 2023-06-27T12:42:46.439 INFO:tasks.rados.rados.0.smithi138.stdout:1119: write oid 1 current snap is 127 2023-06-27T12:42:46.439 INFO:tasks.rados.rados.0.smithi138.stdout:1119: seq_num 542 ranges {709394=646021,2083358=718180} 2023-06-27T12:42:46.448 INFO:tasks.rados.rados.0.smithi138.stdout:1119: writing smithi138138488-1 from 709394 to 1355415 tid 1 2023-06-27T12:42:46.460 INFO:tasks.rados.rados.0.smithi138.stdout:1119: writing smithi138138488-1 from 2083358 to 2801538 tid 2 2023-06-27T12:42:46.460 INFO:tasks.rados.rados.0.smithi138.stdout:1120: delete oid 30 current snap is 127 2023-06-27T12:42:46.518 INFO:tasks.rados.rados.0.smithi138.stdout:1119: finishing write tid 1 to smithi138138488-1 2023-06-27T12:42:46.526 INFO:tasks.rados.rados.0.smithi138.stdout:1119: finishing write tid 2 to smithi138138488-1 2023-06-27T12:42:46.562 INFO:tasks.rados.rados.0.smithi138.stdout:1119: finishing write tid 3 to smithi138138488-1 2023-06-27T12:42:46.562 INFO:tasks.rados.rados.0.smithi138.stdout:1120: done (1 left) 2023-06-27T12:42:46.562 INFO:tasks.rados.rados.0.smithi138.stdout:1121: read oid 24 snap -1 2023-06-27T12:42:46.562 INFO:tasks.rados.rados.0.smithi138.stdout:1121: expect (ObjNum 471 snap 110 seq_num 471) 2023-06-27T12:42:46.562 INFO:tasks.rados.rados.0.smithi138.stdout:1122: snap_remove snap 125 2023-06-27T12:42:46.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:46.920 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:46.915+0000 7f0f7d0f1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:46.923 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:46.919+0000 7f0f7d0f1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:46.969 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:46.972 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:46.973 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:47.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~132s 2023-06-27T12:42:47.442 INFO:tasks.rados.rados.0.smithi138.stdout:1122: done (2 left) 2023-06-27T12:42:47.442 INFO:tasks.rados.rados.0.smithi138.stdout:1123: snap_create 2023-06-27T12:42:47.442 INFO:tasks.rados.rados.0.smithi138.stdout:1119: finishing write tid 4 to smithi138138488-1 2023-06-27T12:42:47.442 INFO:tasks.rados.rados.0.smithi138.stdout:1119: oid 1 updating version 0 to 316 2023-06-27T12:42:47.442 INFO:tasks.rados.rados.0.smithi138.stdout:1119: oid 1 version 316 is already newer than 316 2023-06-27T12:42:47.442 INFO:tasks.rados.rados.0.smithi138.stdout:1119: oid 1 version 316 is already newer than 316 2023-06-27T12:42:47.443 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 316 (ObjNum 542 snap 127 seq_num 542) dirty exists 2023-06-27T12:42:47.443 INFO:tasks.rados.rados.0.smithi138.stdout:1119: left oid 1 (ObjNum 542 snap 127 seq_num 542) 2023-06-27T12:42:47.443 INFO:tasks.rados.rados.0.smithi138.stdout:1121: expect (ObjNum 471 snap 110 seq_num 471) 2023-06-27T12:42:48.442 INFO:tasks.rados.rados.0.smithi138.stdout:1119: done (2 left) 2023-06-27T12:42:48.443 INFO:tasks.rados.rados.0.smithi138.stdout:1121: done (1 left) 2023-06-27T12:42:48.443 INFO:tasks.rados.rados.0.smithi138.stdout:1123: done (0 left) 2023-06-27T12:42:48.443 INFO:tasks.rados.rados.0.smithi138.stdout:1124: write oid 48 current snap is 128 2023-06-27T12:42:48.443 INFO:tasks.rados.rados.0.smithi138.stdout:1124: seq_num 544 ranges {743008=796519,2200322=721849} 2023-06-27T12:42:48.450 INFO:tasks.rados.rados.0.smithi138.stdout:1124: writing smithi138138488-48 from 743008 to 1539527 tid 1 2023-06-27T12:42:48.459 INFO:tasks.rados.rados.0.smithi138.stdout:1124: writing smithi138138488-48 from 2200322 to 2922171 tid 2 2023-06-27T12:42:48.459 INFO:tasks.rados.rados.0.smithi138.stdout:1125: snap_remove snap 127 2023-06-27T12:42:48.745 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:42:48.745 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:42:48.745 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:48.883 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:48.875+0000 7f28ba484700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:48.886 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:48.882+0000 7f28ba484700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:48.974 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:49.089 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:49.101 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2023-06-27T12:42:48.437370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97920750,"num_objects":266,"num_object_clones":220,"num_object_copies":798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1627,"num_read_kb":1167225,"num_write":1431,"num_write_kb":413961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2670,"ondisk_log_size":2670,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":837296,"kb_used_data":701328,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376650064,"statfs":{"total":386547056640,"available":385689665536,"internally_reserved":0,"allocated":718159872,"data_stored":806266753,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.840281"},"pg_stats":[{"pgid":"3.f","version":"256'32","reported_seq":641,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:42.605418+0000","last_change":"2023-06-27T12:42:42.607198+0000","last_active":"2023-06-27T12:42:42.605418+0000","last_peered":"2023-06-27T12:42:42.605418+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:42.605418+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2689219,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":10,"num_read_kb":5,"num_write":23,"num_write_kb":6052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"262'161","reported_seq":849,"reported_epoch":270,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:43.606876+0000","last_change":"2023-06-27T12:42:42.650280+0000","last_active":"2023-06-27T12:42:43.606876+0000","last_peered":"2023-06-27T12:42:43.606876+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:43.606876+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9428958,"num_objects":21,"num_object_clones":17,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":122,"num_read_kb":147869,"num_write":114,"num_write_kb":33537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"264'209","reported_seq":841,"reported_epoch":267,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:42.634832+0000","last_change":"2023-06-27T12:42:42.638448+0000","last_active":"2023-06-27T12:42:42.634832+0000","last_peered":"2023-06-27T12:42:42.634832+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:42.634832+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14168697,"num_objects":24,"num_object_clones":19,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":92611,"num_write":150,"num_write_kb":43261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"268'111","reported_seq":781,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:42.668824+0000","last_change":"2023-06-27T12:42:42.669271+0000","last_active":"2023-06-27T12:42:42.668824+0000","last_peered":"2023-06-27T12:42:42.668824+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:42.668824+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9529833,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":100,"num_read_kb":54528,"num_write":88,"num_write_kb":24907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"263'73","reported_seq":346,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:42.598223+0000","last_change":"2023-06-27T12:42:42.601445+0000","last_active":"2023-06-27T12:42:42.598223+0000","last_peered":"2023-06-27T12:42:42.598223+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:42.598223+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2907710,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":69,"num_read_kb":34136,"num_write":34,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"264'337","reported_seq":870,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-06-27T12:42:42.605425+0000","last_change":"2023-06-27T12:42:42.663696+0000","last_active":"2023-06-27T12:42:42.605425+0000","last_peered":"2023-06-27T12:42:42.605425+0000","last_clean":"2023-06-27T12:42:42.605425+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:42.605425+0000","last_undegraded":"2023-06-27T12:42:42.605425+0000","last_fullsized":"2023-06-27T12:42:42.605425+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14420868,"num_objects":37,"num_object_clones":31,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":266,"num_read_kb":166257,"num_write":205,"num_write_kb":59070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"249'154","reported_seq":691,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-06-27T12:42:42.598846+0000","last_change":"2023-06-27T12:42:42.624063+0000","last_active":"2023-06-27T12:42:42.598846+0000","last_peered":"2023-06-27T12:42:42.598846+0000","last_clean":"2023-06-27T12:42:42.598846+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:42.598846+0000","last_undegraded":"2023-06-27T12:42:42.598846+0000","last_fullsized":"2023-06-27T12:42:42.598846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":110,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":690,"reported_epoch":267,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:40.594519+0000","last_change":"2023-06-27T12:42:40.596261+0000","last_active":"2023-06-27T12:42:40.594519+0000","last_peered":"2023-06-27T12:42:40.594519+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:40.594519+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":444,"reported_epoch":270,"state":"active+undersized","last_fresh":"2023-06-27T12:42:43.608978+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:43.608978+0000","last_peered":"2023-06-27T12:42:43.608978+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:43.608978+0000","last_undegraded":"2023-06-27T12:42:43.608978+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":604,"reported_epoch":267,"state":"active+undersized","last_fresh":"2023-06-27T12:42:40.598146+0000","last_change":"2023-06-27T12:42:40.602612+0000","last_active":"2023-06-27T12:42:40.598146+0000","last_peered":"2023-06-27T12:42:40.598146+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:40.598146+0000","last_undegraded":"2023-06-27T12:42:40.598146+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":425,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-06-27T12:42:42.600198+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:42.600198+0000","last_peered":"2023-06-27T12:42:42.600198+0000","last_clean":"2023-06-27T12:42:42.600198+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:42.600198+0000","last_undegraded":"2023-06-27T12:42:42.600198+0000","last_fullsized":"2023-06-27T12:42:42.600198+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"261'150","reported_seq":787,"reported_epoch":270,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:43.610127+0000","last_change":"2023-06-27T12:42:42.672595+0000","last_active":"2023-06-27T12:42:43.610127+0000","last_peered":"2023-06-27T12:42:43.610127+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:43.610127+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563315,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":66415,"num_write":101,"num_write_kb":27991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":418,"reported_epoch":267,"state":"active+undersized","last_fresh":"2023-06-27T12:42:40.588626+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:40.588626+0000","last_peered":"2023-06-27T12:42:40.588626+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:40.588626+0000","last_undegraded":"2023-06-27T12:42:40.588626+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"268'343","reported_seq":419,"reported_epoch":270,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:43.603314+0000","last_change":"2023-06-27T12:42:42.652295+0000","last_active":"2023-06-27T12:42:43.603314+0000","last_peered":"2023-06-27T12:42:43.603314+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:43.603314+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6487733,"num_objects":12,"num_object_clones":11,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":41,"num_read_kb":24503,"num_write":68,"num_write_kb":20973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":415,"reported_epoch":267,"state":"active+clean","last_fresh":"2023-06-27T12:42:40.595205+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:40.595205+0000","last_peered":"2023-06-27T12:42:40.595205+0000","last_clean":"2023-06-27T12:42:40.595205+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:40.595205+0000","last_undegraded":"2023-06-27T12:42:40.595205+0000","last_fullsized":"2023-06-27T12:42:40.595205+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":177,"reported_epoch":269,"state":"active+undersized","last_fresh":"2023-06-27T12:42:42.613052+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:42.613052+0000","last_peered":"2023-06-27T12:42:42.613052+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:42.613052+0000","last_undegraded":"2023-06-27T12:42:42.613052+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"263'310","reported_seq":971,"reported_epoch":267,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:42.629634+0000","last_change":"2023-06-27T12:42:42.631260+0000","last_active":"2023-06-27T12:42:42.629634+0000","last_peered":"2023-06-27T12:42:42.629634+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:42.629634+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8151692,"num_objects":41,"num_object_clones":35,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":277,"num_read_kb":173895,"num_write":228,"num_write_kb":65793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":41}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":425,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-06-27T12:42:42.596440+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:42.596440+0000","last_peered":"2023-06-27T12:42:42.596440+0000","last_clean":"2023-06-27T12:42:42.596440+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:42.596440+0000","last_undegraded":"2023-06-27T12:42:42.596440+0000","last_fullsized":"2023-06-27T12:42:42.596440+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"261'167","reported_seq":364,"reported_epoch":267,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:42.634091+0000","last_change":"2023-06-27T12:42:42.637746+0000","last_active":"2023-06-27T12:42:42.634091+0000","last_peered":"2023-06-27T12:42:42.634091+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:42.634091+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722866,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":74,"num_read_kb":41526,"num_write":49,"num_write_kb":13967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":187,"reported_epoch":270,"state":"active+undersized","last_fresh":"2023-06-27T12:42:43.603913+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:43.603913+0000","last_peered":"2023-06-27T12:42:43.603913+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:43.603913+0000","last_undegraded":"2023-06-27T12:42:43.603913+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"269'169","reported_seq":820,"reported_epoch":270,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:43.610614+0000","last_change":"2023-06-27T12:42:42.628052+0000","last_active":"2023-06-27T12:42:43.610614+0000","last_peered":"2023-06-27T12:42:43.610614+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:43.610614+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714710,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":93,"num_read_kb":105200,"num_write":114,"num_write_kb":33633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":357,"reported_epoch":267,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:40.597574+0000","last_change":"2023-06-27T12:42:40.600651+0000","last_active":"2023-06-27T12:42:40.597574+0000","last_peered":"2023-06-27T12:42:40.597574+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:40.597574+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":42747,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":444,"reported_epoch":270,"state":"active+undersized","last_fresh":"2023-06-27T12:42:43.610274+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:43.610274+0000","last_peered":"2023-06-27T12:42:43.610274+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:43.610274+0000","last_undegraded":"2023-06-27T12:42:43.610274+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":415,"reported_epoch":267,"state":"active+clean","last_fresh":"2023-06-27T12:42:40.592666+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:40.592666+0000","last_peered":"2023-06-27T12:42:40.592666+0000","last_clean":"2023-06-27T12:42:40.592666+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:40.592666+0000","last_undegraded":"2023-06-27T12:42:40.592666+0000","last_fullsized":"2023-06-27T12:42:40.592666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"268'186","reported_seq":807,"reported_epoch":270,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:43.609180+0000","last_change":"2023-06-27T12:42:42.680308+0000","last_active":"2023-06-27T12:42:43.609180+0000","last_peered":"2023-06-27T12:42:43.609180+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:43.609180+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294403,"num_objects":20,"num_object_clones":17,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":93,"num_read_kb":92051,"num_write":97,"num_write_kb":28022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":97920750,"num_objects":266,"num_object_clones":220,"num_object_copies":798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1627,"num_read_kb":1167225,"num_write":1431,"num_write_kb":413961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2573,"ondisk_log_size":2573,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":206224,"kb_used_data":171208,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94165616,"statfs":{"total":96636764160,"available":96425590784,"internally_reserved":0,"allocated":175316992,"data_stored":197094335,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":218052,"kb_used_data":183036,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94153788,"statfs":{"total":96636764160,"available":96413478912,"internally_reserved":0,"allocated":187428864,"data_stored":207817125,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":231740,"kb_used_data":196724,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94140100,"statfs":{"total":96636764160,"available":96399462400,"internally_reserved":0,"allocated":201445376,"data_stored":224315741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:49.103 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.8 3.3 3.1 3.4 3.7 3.6 2023-06-27T12:42:49.104 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:49.096+0000 7f40bf6ae700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:49.104 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:49.100+0000 7f40bf6ae700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:49.159 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:49.163 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:49.163 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:49.243 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:49.228+0000 7f79ef313700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:49.246 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:49.242+0000 7f79ef313700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:49.438 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.f] on osd.0 to force-recovery; instructing pg(s) [3.1,3.7,3.8] on osd.1 to force-recovery; instructing pg(s) [3.3,3.4,3.6,3.e] on osd.2 to force-recovery; 2023-06-27T12:42:49.438 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:42:49.457 INFO:tasks.rados.rados.0.smithi138.stdout:1125: done (1 left) 2023-06-27T12:42:49.458 INFO:tasks.rados.rados.0.smithi138.stdout:1126: read oid 12 snap 126 2023-06-27T12:42:49.458 INFO:tasks.rados.rados.0.smithi138.stdout:1126: expect (ObjNum 541 snap 126 seq_num 541) 2023-06-27T12:42:49.458 INFO:tasks.rados.rados.0.smithi138.stdout:1127: write oid 38 current snap is 128 2023-06-27T12:42:49.459 INFO:tasks.rados.rados.0.smithi138.stdout:1124: finishing write tid 1 to smithi138138488-48 2023-06-27T12:42:49.459 INFO:tasks.rados.rados.0.smithi138.stdout:1124: finishing write tid 2 to smithi138138488-48 2023-06-27T12:42:49.459 INFO:tasks.rados.rados.0.smithi138.stdout:1124: finishing write tid 3 to smithi138138488-48 2023-06-27T12:42:49.459 INFO:tasks.rados.rados.0.smithi138.stdout:1127: seq_num 545 ranges {759815=471768,1742451=465507,2917178=242637} 2023-06-27T12:42:49.466 INFO:tasks.rados.rados.0.smithi138.stdout:1127: writing smithi138138488-38 from 759815 to 1231583 tid 1 2023-06-27T12:42:49.473 INFO:tasks.rados.rados.0.smithi138.stdout:1127: writing smithi138138488-38 from 1742451 to 2207958 tid 2 2023-06-27T12:42:49.480 INFO:tasks.rados.rados.0.smithi138.stdout:1127: writing smithi138138488-38 from 2917178 to 3159815 tid 3 2023-06-27T12:42:49.481 INFO:tasks.rados.rados.0.smithi138.stdout:1128: delete oid 22 current snap is 128 2023-06-27T12:42:49.481 INFO:tasks.rados.rados.0.smithi138.stdout:1124: finishing write tid 4 to smithi138138488-48 2023-06-27T12:42:49.481 INFO:tasks.rados.rados.0.smithi138.stdout:1124: oid 48 updating version 0 to 192 2023-06-27T12:42:49.481 INFO:tasks.rados.rados.0.smithi138.stdout:1124: oid 48 version 192 is already newer than 192 2023-06-27T12:42:49.481 INFO:tasks.rados.rados.0.smithi138.stdout:1124: oid 48 version 192 is already newer than 192 2023-06-27T12:42:49.481 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 192 (ObjNum 544 snap 128 seq_num 544) dirty exists 2023-06-27T12:42:49.481 INFO:tasks.rados.rados.0.smithi138.stdout:1124: left oid 48 (ObjNum 544 snap 128 seq_num 544) 2023-06-27T12:42:49.582 INFO:tasks.rados.rados.0.smithi138.stdout:1124: done (3 left) 2023-06-27T12:42:49.583 INFO:tasks.rados.rados.0.smithi138.stdout:1128: done (2 left) 2023-06-27T12:42:49.583 INFO:tasks.rados.rados.0.smithi138.stdout:1129: snap_remove snap 123 2023-06-27T12:42:50.464 INFO:tasks.rados.rados.0.smithi138.stdout:1129: done (2 left) 2023-06-27T12:42:50.464 INFO:tasks.rados.rados.0.smithi138.stdout:1130: write (excl) oid 22 current snap is 128 2023-06-27T12:42:50.464 INFO:tasks.rados.rados.0.smithi138.stdout:1130: seq_num 547 ranges {793429=622266,1859415=469176,2972024=221405} 2023-06-27T12:42:50.474 INFO:tasks.rados.rados.0.smithi138.stdout:1130: writing smithi138138488-22 from 793429 to 1415695 tid 1 2023-06-27T12:42:50.480 INFO:tasks.rados.rados.0.smithi138.stdout:1130: writing smithi138138488-22 from 1859415 to 2328591 tid 2 2023-06-27T12:42:50.487 INFO:tasks.rados.rados.0.smithi138.stdout:1130: writing smithi138138488-22 from 2972024 to 3193429 tid 3 2023-06-27T12:42:50.487 INFO:tasks.rados.rados.0.smithi138.stdout:1131: read oid 5 snap 113 2023-06-27T12:42:50.487 INFO:tasks.rados.rados.0.smithi138.stdout:1131: expect deleted 2023-06-27T12:42:50.488 INFO:tasks.rados.rados.0.smithi138.stdout:1132: snap_create 2023-06-27T12:42:50.488 INFO:tasks.rados.rados.0.smithi138.stdout:1127: finishing write tid 1 to smithi138138488-38 2023-06-27T12:42:50.488 INFO:tasks.rados.rados.0.smithi138.stdout:1126: expect (ObjNum 541 snap 126 seq_num 541) 2023-06-27T12:42:50.738 INFO:tasks.rados.rados.0.smithi138.stdout:1127: finishing write tid 2 to smithi138138488-38 2023-06-27T12:42:50.738 INFO:tasks.rados.rados.0.smithi138.stdout:1127: finishing write tid 3 to smithi138138488-38 2023-06-27T12:42:50.739 INFO:tasks.rados.rados.0.smithi138.stdout:1127: finishing write tid 4 to smithi138138488-38 2023-06-27T12:42:50.739 INFO:tasks.rados.rados.0.smithi138.stdout:1127: finishing write tid 5 to smithi138138488-38 2023-06-27T12:42:50.739 INFO:tasks.rados.rados.0.smithi138.stdout:1127: oid 38 updating version 0 to 37 2023-06-27T12:42:50.739 INFO:tasks.rados.rados.0.smithi138.stdout:1127: oid 38 version 37 is already newer than 37 2023-06-27T12:42:50.739 INFO:tasks.rados.rados.0.smithi138.stdout:1127: oid 38 version 37 is already newer than 37 2023-06-27T12:42:50.739 INFO:tasks.rados.rados.0.smithi138.stdout:1127: oid 38 version 37 is already newer than 37 2023-06-27T12:42:50.739 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 37 (ObjNum 545 snap 128 seq_num 545) dirty exists 2023-06-27T12:42:50.739 INFO:tasks.rados.rados.0.smithi138.stdout:1127: left oid 38 (ObjNum 545 snap 128 seq_num 545) 2023-06-27T12:42:50.740 INFO:tasks.rados.rados.0.smithi138.stdout:1130: finishing write tid 1 to smithi138138488-22 2023-06-27T12:42:50.740 INFO:tasks.rados.rados.0.smithi138.stdout:1130: finishing write tid 2 to smithi138138488-22 2023-06-27T12:42:50.740 INFO:tasks.rados.rados.0.smithi138.stdout:1130: finishing write tid 3 to smithi138138488-22 2023-06-27T12:42:50.740 INFO:tasks.rados.rados.0.smithi138.stdout:1130: finishing write tid 4 to smithi138138488-22 2023-06-27T12:42:50.740 INFO:tasks.rados.rados.0.smithi138.stdout:1130: finishing write tid 5 to smithi138138488-22 2023-06-27T12:42:50.740 INFO:tasks.rados.rados.0.smithi138.stdout:1130: oid 22 updating version 0 to 202 2023-06-27T12:42:50.740 INFO:tasks.rados.rados.0.smithi138.stdout:1130: oid 22 version 202 is already newer than 202 2023-06-27T12:42:50.741 INFO:tasks.rados.rados.0.smithi138.stdout:1130: oid 22 version 202 is already newer than 202 2023-06-27T12:42:50.741 INFO:tasks.rados.rados.0.smithi138.stdout:1130: oid 22 version 202 is already newer than 199 2023-06-27T12:42:50.741 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 202 (ObjNum 547 snap 128 seq_num 547) dirty exists 2023-06-27T12:42:50.741 INFO:tasks.rados.rados.0.smithi138.stdout:1130: left oid 22 (ObjNum 547 snap 128 seq_num 547) 2023-06-27T12:42:51.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:51.291 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:51.286+0000 7f747a599700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:51.294 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:51.290+0000 7f747a599700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:51.339 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:51.342 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:51.343 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:51.464 INFO:tasks.rados.rados.0.smithi138.stdout:1126: done (4 left) 2023-06-27T12:42:51.464 INFO:tasks.rados.rados.0.smithi138.stdout:1127: done (3 left) 2023-06-27T12:42:51.464 INFO:tasks.rados.rados.0.smithi138.stdout:1130: done (2 left) 2023-06-27T12:42:51.465 INFO:tasks.rados.rados.0.smithi138.stdout:1131: done (1 left) 2023-06-27T12:42:51.465 INFO:tasks.rados.rados.0.smithi138.stdout:1132: done (0 left) 2023-06-27T12:42:51.465 INFO:tasks.rados.rados.0.smithi138.stdout:1133: snap_create 2023-06-27T12:42:52.466 INFO:tasks.rados.rados.0.smithi138.stdout:1133: done (0 left) 2023-06-27T12:42:52.466 INFO:tasks.rados.rados.0.smithi138.stdout:1134: write oid 36 current snap is 130 2023-06-27T12:42:52.466 INFO:tasks.rados.rados.0.smithi138.stdout:1134: seq_num 548 ranges {410236=413868,1234250=729187,2715800=423063} 2023-06-27T12:42:52.471 INFO:tasks.rados.rados.0.smithi138.stdout:1134: writing smithi138138488-36 from 410236 to 824104 tid 1 2023-06-27T12:42:52.481 INFO:tasks.rados.rados.0.smithi138.stdout:1134: writing smithi138138488-36 from 1234250 to 1963437 tid 2 2023-06-27T12:42:52.490 INFO:tasks.rados.rados.0.smithi138.stdout:1134: writing smithi138138488-36 from 2715800 to 3138863 tid 3 2023-06-27T12:42:52.490 INFO:tasks.rados.rados.0.smithi138.stdout:1135: write oid 5 current snap is 130 2023-06-27T12:42:52.490 INFO:tasks.rados.rados.0.smithi138.stdout:1135: seq_num 549 ranges {427043=489117,1576379=472845,2626870=600173} 2023-06-27T12:42:52.496 INFO:tasks.rados.rados.0.smithi138.stdout:1135: writing smithi138138488-5 from 427043 to 916160 tid 1 2023-06-27T12:42:52.505 INFO:tasks.rados.rados.0.smithi138.stdout:1135: writing smithi138138488-5 from 1576379 to 2049224 tid 2 2023-06-27T12:42:52.514 INFO:tasks.rados.rados.0.smithi138.stdout:1135: writing smithi138138488-5 from 2626870 to 3227043 tid 3 2023-06-27T12:42:52.514 INFO:tasks.rados.rados.0.smithi138.stdout:1136: snap_create 2023-06-27T12:42:52.594 INFO:tasks.rados.rados.0.smithi138.stdout:1134: finishing write tid 1 to smithi138138488-36 2023-06-27T12:42:52.604 INFO:tasks.rados.rados.0.smithi138.stdout:1134: finishing write tid 2 to smithi138138488-36 2023-06-27T12:42:52.639 INFO:tasks.rados.rados.0.smithi138.stdout:1134: finishing write tid 3 to smithi138138488-36 2023-06-27T12:42:52.652 INFO:tasks.rados.rados.0.smithi138.stdout:1134: finishing write tid 4 to smithi138138488-36 2023-06-27T12:42:52.654 INFO:tasks.rados.rados.0.smithi138.stdout:1134: finishing write tid 5 to smithi138138488-36 2023-06-27T12:42:52.655 INFO:tasks.rados.rados.0.smithi138.stdout:1134: oid 36 updating version 0 to 231 2023-06-27T12:42:52.655 INFO:tasks.rados.rados.0.smithi138.stdout:1134: oid 36 version 231 is already newer than 231 2023-06-27T12:42:52.655 INFO:tasks.rados.rados.0.smithi138.stdout:1134: oid 36 version 231 is already newer than 231 2023-06-27T12:42:52.655 INFO:tasks.rados.rados.0.smithi138.stdout:1134: oid 36 version 231 is already newer than 230 2023-06-27T12:42:52.655 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 231 (ObjNum 548 snap 130 seq_num 548) dirty exists 2023-06-27T12:42:52.655 INFO:tasks.rados.rados.0.smithi138.stdout:1134: left oid 36 (ObjNum 548 snap 130 seq_num 548) 2023-06-27T12:42:52.661 INFO:tasks.rados.rados.0.smithi138.stdout:1135: finishing write tid 1 to smithi138138488-5 2023-06-27T12:42:52.674 INFO:tasks.rados.rados.0.smithi138.stdout:1135: finishing write tid 2 to smithi138138488-5 2023-06-27T12:42:52.675 INFO:tasks.rados.rados.0.smithi138.stdout:1135: finishing write tid 3 to smithi138138488-5 2023-06-27T12:42:52.677 INFO:tasks.rados.rados.0.smithi138.stdout:1135: finishing write tid 4 to smithi138138488-5 2023-06-27T12:42:52.678 INFO:tasks.rados.rados.0.smithi138.stdout:1135: finishing write tid 5 to smithi138138488-5 2023-06-27T12:42:52.679 INFO:tasks.rados.rados.0.smithi138.stdout:1135: oid 5 updating version 0 to 188 2023-06-27T12:42:52.679 INFO:tasks.rados.rados.0.smithi138.stdout:1135: oid 5 updating version 188 to 189 2023-06-27T12:42:52.679 INFO:tasks.rados.rados.0.smithi138.stdout:1135: oid 5 version 189 is already newer than 189 2023-06-27T12:42:52.679 INFO:tasks.rados.rados.0.smithi138.stdout:1135: oid 5 version 189 is already newer than 189 2023-06-27T12:42:52.680 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 5 v 189 (ObjNum 549 snap 130 seq_num 549) dirty exists 2023-06-27T12:42:52.680 INFO:tasks.rados.rados.0.smithi138.stdout:1135: left oid 5 (ObjNum 549 snap 130 seq_num 549) 2023-06-27T12:42:52.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~138s 2023-06-27T12:42:53.344 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:53.473 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:53.468+0000 7f30ca509700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:53.477 INFO:tasks.rados.rados.0.smithi138.stdout:1134: done (2 left) 2023-06-27T12:42:53.477 INFO:tasks.rados.rados.0.smithi138.stdout:1135: done (1 left) 2023-06-27T12:42:53.478 INFO:tasks.rados.rados.0.smithi138.stdout:1136: done (0 left) 2023-06-27T12:42:53.478 INFO:tasks.rados.rados.0.smithi138.stdout:1137: write oid 34 current snap is 131 2023-06-27T12:42:53.478 INFO:tasks.rados.rados.0.smithi138.stdout:1137: seq_num 550 ranges {443850=564366,1634861=732856,2770646=473204} 2023-06-27T12:42:53.479 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:53.474+0000 7f30ca509700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:53.483 INFO:tasks.rados.rados.0.smithi138.stdout:1137: writing smithi138138488-34 from 443850 to 1008216 tid 1 2023-06-27T12:42:53.492 INFO:tasks.rados.rados.0.smithi138.stdout:1137: writing smithi138138488-34 from 1634861 to 2367717 tid 2 2023-06-27T12:42:53.499 INFO:tasks.rados.rados.0.smithi138.stdout:1137: writing smithi138138488-34 from 2770646 to 3243850 tid 3 2023-06-27T12:42:53.499 INFO:tasks.rados.rados.0.smithi138.stdout:1138: snap_create 2023-06-27T12:42:53.553 INFO:tasks.rados.rados.0.smithi138.stdout:1137: finishing write tid 1 to smithi138138488-34 2023-06-27T12:42:53.554 INFO:tasks.rados.rados.0.smithi138.stdout:1137: finishing write tid 2 to smithi138138488-34 2023-06-27T12:42:53.555 INFO:tasks.rados.rados.0.smithi138.stdout:1137: finishing write tid 3 to smithi138138488-34 2023-06-27T12:42:53.562 INFO:tasks.rados.rados.0.smithi138.stdout:1137: finishing write tid 4 to smithi138138488-34 2023-06-27T12:42:53.563 INFO:tasks.rados.rados.0.smithi138.stdout:1137: finishing write tid 5 to smithi138138488-34 2023-06-27T12:42:53.563 INFO:tasks.rados.rados.0.smithi138.stdout:1137: oid 34 updating version 0 to 346 2023-06-27T12:42:53.564 INFO:tasks.rados.rados.0.smithi138.stdout:1137: oid 34 version 346 is already newer than 346 2023-06-27T12:42:53.564 INFO:tasks.rados.rados.0.smithi138.stdout:1137: oid 34 version 346 is already newer than 346 2023-06-27T12:42:53.564 INFO:tasks.rados.rados.0.smithi138.stdout:1137: oid 34 version 346 is already newer than 346 2023-06-27T12:42:53.565 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 346 (ObjNum 550 snap 131 seq_num 550) dirty exists 2023-06-27T12:42:53.565 INFO:tasks.rados.rados.0.smithi138.stdout:1137: left oid 34 (ObjNum 550 snap 131 seq_num 550) 2023-06-27T12:42:53.576 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:53.579 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:53.580 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:54.449 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:42:54.450 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-06-27T12:42:54.481 INFO:tasks.rados.rados.0.smithi138.stdout:1137: done (1 left) 2023-06-27T12:42:54.482 INFO:tasks.rados.rados.0.smithi138.stdout:1138: done (0 left) 2023-06-27T12:42:54.482 INFO:tasks.rados.rados.0.smithi138.stdout:1139: snap_remove snap 126 2023-06-27T12:42:54.591 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:54.582+0000 7f8b7b7d3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:54.594 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:54.590+0000 7f8b7b7d3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:55.438 INFO:tasks.rados.rados.0.smithi138.stdout:1139: done (0 left) 2023-06-27T12:42:55.438 INFO:tasks.rados.rados.0.smithi138.stdout:1140: read oid 44 snap -1 2023-06-27T12:42:55.438 INFO:tasks.rados.rados.0.smithi138.stdout:1140: expect (ObjNum 531 snap 122 seq_num 531) 2023-06-27T12:42:55.438 INFO:tasks.rados.rados.0.smithi138.stdout:1141: write oid 12 current snap is 132 2023-06-27T12:42:55.438 INFO:tasks.rados.rados.0.smithi138.stdout:1141: seq_num 551 ranges {460657=639615,1693343=476514,2681716=578941} 2023-06-27T12:42:55.440 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 1 (10000) 2023-06-27T12:42:55.446 INFO:tasks.rados.rados.0.smithi138.stdout:1141: writing smithi138138488-12 from 460657 to 1100272 tid 1 2023-06-27T12:42:55.454 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-06-27T12:42:55.455 INFO:tasks.rados.rados.0.smithi138.stdout:1141: writing smithi138138488-12 from 1693343 to 2169857 tid 2 2023-06-27T12:42:55.463 INFO:tasks.rados.rados.0.smithi138.stdout:1141: writing smithi138138488-12 from 2681716 to 3260657 tid 3 2023-06-27T12:42:55.464 INFO:tasks.rados.rados.0.smithi138.stdout:1142: read oid 25 snap -1 2023-06-27T12:42:55.464 INFO:tasks.rados.rados.0.smithi138.stdout:1142: expect (ObjNum 501 snap 114 seq_num 501) 2023-06-27T12:42:55.464 INFO:tasks.rados.rados.0.smithi138.stdout:1143: read oid 36 snap -1 2023-06-27T12:42:55.464 INFO:tasks.rados.rados.0.smithi138.stdout:1143: expect (ObjNum 548 snap 130 seq_num 548) 2023-06-27T12:42:55.464 INFO:tasks.rados.rados.0.smithi138.stdout:1144: write oid 9 current snap is 132 2023-06-27T12:42:55.464 INFO:tasks.rados.rados.0.smithi138.stdout:1144: seq_num 552 ranges {477464=714864,1751825=736525,3225492=51972} 2023-06-27T12:42:55.472 INFO:tasks.rados.rados.0.smithi138.stdout:1144: writing smithi138138488-9 from 477464 to 1192328 tid 1 2023-06-27T12:42:55.482 INFO:tasks.rados.rados.0.smithi138.stdout:1144: writing smithi138138488-9 from 1751825 to 2488350 tid 2 2023-06-27T12:42:55.487 INFO:tasks.rados.rados.0.smithi138.stdout:1144: writing smithi138138488-9 from 3225492 to 3277464 tid 3 2023-06-27T12:42:55.488 INFO:tasks.rados.rados.0.smithi138.stdout:1145: read oid 22 snap 130 2023-06-27T12:42:55.488 INFO:tasks.rados.rados.0.smithi138.stdout:1145: expect (ObjNum 547 snap 128 seq_num 547) 2023-06-27T12:42:55.488 INFO:tasks.rados.rados.0.smithi138.stdout:1146: write (excl) oid 15 current snap is 132 2023-06-27T12:42:55.488 INFO:tasks.rados.rados.0.smithi138.stdout:1146: seq_num 553 ranges {494271=790113,1693954=480183,2620209=512129} 2023-06-27T12:42:55.498 INFO:tasks.rados.rados.0.smithi138.stdout:1146: writing smithi138138488-15 from 494271 to 1284384 tid 1 2023-06-27T12:42:55.505 INFO:tasks.rados.rados.0.smithi138.stdout:1146: writing smithi138138488-15 from 1693954 to 2174137 tid 2 2023-06-27T12:42:55.512 INFO:tasks.rados.rados.0.smithi138.stdout:1146: writing smithi138138488-15 from 2620209 to 3132338 tid 3 2023-06-27T12:42:55.513 INFO:tasks.rados.rados.0.smithi138.stdout:1147: read oid 48 snap -1 2023-06-27T12:42:55.513 INFO:tasks.rados.rados.0.smithi138.stdout:1147: expect (ObjNum 544 snap 128 seq_num 544) 2023-06-27T12:42:55.513 INFO:tasks.rados.rados.0.smithi138.stdout:1148: write oid 35 current snap is 132 2023-06-27T12:42:55.513 INFO:tasks.rados.rados.0.smithi138.stdout:1148: seq_num 554 ranges {511078=465362,1752436=740194,3163985=147093} 2023-06-27T12:42:55.520 INFO:tasks.rados.rados.0.smithi138.stdout:1148: writing smithi138138488-35 from 511078 to 976440 tid 1 2023-06-27T12:42:55.531 INFO:tasks.rados.rados.0.smithi138.stdout:1148: writing smithi138138488-35 from 1752436 to 2492630 tid 2 2023-06-27T12:42:55.537 INFO:tasks.rados.rados.0.smithi138.stdout:1148: writing smithi138138488-35 from 3163985 to 3311078 tid 3 2023-06-27T12:42:55.537 INFO:tasks.rados.rados.0.smithi138.stdout:1149: write oid 20 current snap is 132 2023-06-27T12:42:55.537 INFO:tasks.rados.rados.0.smithi138.stdout:1149: seq_num 555 ranges {527885=656964,1927271=483852,3191408=136477} 2023-06-27T12:42:55.546 INFO:tasks.rados.rados.0.smithi138.stdout:1149: writing smithi138138488-20 from 527885 to 1184849 tid 1 2023-06-27T12:42:55.555 INFO:tasks.rados.rados.0.smithi138.stdout:1149: writing smithi138138488-20 from 1927271 to 2411123 tid 2 2023-06-27T12:42:55.561 INFO:tasks.rados.rados.0.smithi138.stdout:1149: writing smithi138138488-20 from 3191408 to 3327885 tid 3 2023-06-27T12:42:55.562 INFO:tasks.rados.rados.0.smithi138.stdout:1150: delete oid 13 current snap is 132 2023-06-27T12:42:55.562 INFO:tasks.rados.rados.0.smithi138.stdout:1140: expect (ObjNum 531 snap 122 seq_num 531) 2023-06-27T12:42:55.581 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:42:55.614 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:55.603+0000 7fa8a640a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:55.620 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:55.615+0000 7fa8a640a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:55.694 INFO:tasks.rados.rados.0.smithi138.stdout:1146: finishing write tid 1 to smithi138138488-15 2023-06-27T12:42:55.694 INFO:tasks.rados.rados.0.smithi138.stdout:1148: finishing write tid 1 to smithi138138488-35 2023-06-27T12:42:55.694 INFO:tasks.rados.rados.0.smithi138.stdout:1146: finishing write tid 2 to smithi138138488-15 2023-06-27T12:42:55.694 INFO:tasks.rados.rados.0.smithi138.stdout:1146: finishing write tid 3 to smithi138138488-15 2023-06-27T12:42:55.695 INFO:tasks.rados.rados.0.smithi138.stdout:1146: finishing write tid 4 to smithi138138488-15 2023-06-27T12:42:55.695 INFO:tasks.rados.rados.0.smithi138.stdout:1146: finishing write tid 5 to smithi138138488-15 2023-06-27T12:42:55.695 INFO:tasks.rados.rados.0.smithi138.stdout:1146: oid 15 updating version 0 to 74 2023-06-27T12:42:55.695 INFO:tasks.rados.rados.0.smithi138.stdout:1146: oid 15 updating version 74 to 77 2023-06-27T12:42:55.695 INFO:tasks.rados.rados.0.smithi138.stdout:1146: oid 15 version 77 is already newer than 77 2023-06-27T12:42:55.695 INFO:tasks.rados.rados.0.smithi138.stdout:1146: oid 15 version 77 is already newer than 77 2023-06-27T12:42:55.695 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 77 (ObjNum 553 snap 132 seq_num 553) dirty exists 2023-06-27T12:42:55.696 INFO:tasks.rados.rados.0.smithi138.stdout:1146: left oid 15 (ObjNum 553 snap 132 seq_num 553) 2023-06-27T12:42:55.730 INFO:tasks.rados.rados.0.smithi138.stdout:1141: finishing write tid 1 to smithi138138488-12 2023-06-27T12:42:55.735 INFO:tasks.rados.rados.0.smithi138.stdout:1142: expect (ObjNum 501 snap 114 seq_num 501) 2023-06-27T12:42:55.740 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:55.735+0000 7ff1b2478700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:55.744 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:55.740+0000 7ff1b2478700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:55.835 INFO:tasks.rados.rados.0.smithi138.stdout:1141: finishing write tid 2 to smithi138138488-12 2023-06-27T12:42:55.836 INFO:tasks.rados.rados.0.smithi138.stdout:1148: finishing write tid 2 to smithi138138488-35 2023-06-27T12:42:55.836 INFO:tasks.rados.rados.0.smithi138.stdout:1148: finishing write tid 3 to smithi138138488-35 2023-06-27T12:42:55.836 INFO:tasks.rados.rados.0.smithi138.stdout:1148: finishing write tid 4 to smithi138138488-35 2023-06-27T12:42:55.836 INFO:tasks.rados.rados.0.smithi138.stdout:1148: finishing write tid 5 to smithi138138488-35 2023-06-27T12:42:55.836 INFO:tasks.rados.rados.0.smithi138.stdout:1148: oid 35 updating version 0 to 348 2023-06-27T12:42:55.836 INFO:tasks.rados.rados.0.smithi138.stdout:1148: oid 35 version 348 is already newer than 348 2023-06-27T12:42:55.836 INFO:tasks.rados.rados.0.smithi138.stdout:1148: oid 35 version 348 is already newer than 348 2023-06-27T12:42:55.837 INFO:tasks.rados.rados.0.smithi138.stdout:1148: oid 35 version 348 is already newer than 348 2023-06-27T12:42:55.837 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 348 (ObjNum 554 snap 132 seq_num 554) dirty exists 2023-06-27T12:42:55.837 INFO:tasks.rados.rados.0.smithi138.stdout:1148: left oid 35 (ObjNum 554 snap 132 seq_num 554) 2023-06-27T12:42:55.837 INFO:tasks.rados.rados.0.smithi138.stdout:1149: finishing write tid 1 to smithi138138488-20 2023-06-27T12:42:55.837 INFO:tasks.rados.rados.0.smithi138.stdout:1141: finishing write tid 3 to smithi138138488-12 2023-06-27T12:42:55.837 INFO:tasks.rados.rados.0.smithi138.stdout:1149: finishing write tid 2 to smithi138138488-20 2023-06-27T12:42:55.837 INFO:tasks.rados.rados.0.smithi138.stdout:1149: finishing write tid 3 to smithi138138488-20 2023-06-27T12:42:55.838 INFO:tasks.rados.rados.0.smithi138.stdout:1141: finishing write tid 4 to smithi138138488-12 2023-06-27T12:42:55.838 INFO:tasks.rados.rados.0.smithi138.stdout:1149: finishing write tid 4 to smithi138138488-20 2023-06-27T12:42:55.838 INFO:tasks.rados.rados.0.smithi138.stdout:1149: finishing write tid 5 to smithi138138488-20 2023-06-27T12:42:55.838 INFO:tasks.rados.rados.0.smithi138.stdout:1149: oid 20 updating version 0 to 168 2023-06-27T12:42:55.838 INFO:tasks.rados.rados.0.smithi138.stdout:1149: oid 20 version 168 is already newer than 168 2023-06-27T12:42:55.838 INFO:tasks.rados.rados.0.smithi138.stdout:1149: oid 20 version 168 is already newer than 168 2023-06-27T12:42:55.838 INFO:tasks.rados.rados.0.smithi138.stdout:1149: oid 20 version 168 is already newer than 168 2023-06-27T12:42:55.838 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 168 (ObjNum 555 snap 132 seq_num 555) dirty exists 2023-06-27T12:42:55.839 INFO:tasks.rados.rados.0.smithi138.stdout:1149: left oid 20 (ObjNum 555 snap 132 seq_num 555) 2023-06-27T12:42:55.839 INFO:tasks.rados.rados.0.smithi138.stdout:1141: finishing write tid 5 to smithi138138488-12 2023-06-27T12:42:55.839 INFO:tasks.rados.rados.0.smithi138.stdout:1141: oid 12 updating version 0 to 238 2023-06-27T12:42:55.839 INFO:tasks.rados.rados.0.smithi138.stdout:1141: oid 12 version 238 is already newer than 238 2023-06-27T12:42:55.839 INFO:tasks.rados.rados.0.smithi138.stdout:1141: oid 12 version 238 is already newer than 238 2023-06-27T12:42:55.839 INFO:tasks.rados.rados.0.smithi138.stdout:1141: oid 12 version 238 is already newer than 238 2023-06-27T12:42:55.839 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 238 (ObjNum 551 snap 132 seq_num 551) dirty exists 2023-06-27T12:42:55.839 INFO:tasks.rados.rados.0.smithi138.stdout:1141: left oid 12 (ObjNum 551 snap 132 seq_num 551) 2023-06-27T12:42:55.857 INFO:tasks.rados.rados.0.smithi138.stdout:1144: finishing write tid 1 to smithi138138488-9 2023-06-27T12:42:55.885 INFO:tasks.rados.rados.0.smithi138.stdout:1144: finishing write tid 2 to smithi138138488-9 2023-06-27T12:42:55.900 INFO:tasks.rados.rados.0.smithi138.stdout:1140: done (10 left) 2023-06-27T12:42:55.901 INFO:tasks.rados.rados.0.smithi138.stdout:1141: done (9 left) 2023-06-27T12:42:55.901 INFO:tasks.rados.rados.0.smithi138.stdout:1142: done (8 left) 2023-06-27T12:42:55.901 INFO:tasks.rados.rados.0.smithi138.stdout:1146: done (7 left) 2023-06-27T12:42:55.901 INFO:tasks.rados.rados.0.smithi138.stdout:1148: done (6 left) 2023-06-27T12:42:55.901 INFO:tasks.rados.rados.0.smithi138.stdout:1149: done (5 left) 2023-06-27T12:42:55.901 INFO:tasks.rados.rados.0.smithi138.stdout:1150: done (4 left) 2023-06-27T12:42:55.901 INFO:tasks.rados.rados.0.smithi138.stdout:1151: snap_remove snap 129 2023-06-27T12:42:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:55.926 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:55.927 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:56.489 INFO:tasks.rados.rados.0.smithi138.stdout:1151: done (4 left) 2023-06-27T12:42:56.489 INFO:tasks.rados.rados.0.smithi138.stdout:1152: write (excl) oid 19 current snap is 132 2023-06-27T12:42:56.489 INFO:tasks.rados.rados.0.smithi138.stdout:1143: expect (ObjNum 548 snap 130 seq_num 548) 2023-06-27T12:42:56.493 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.1 to 1 (10000) 2023-06-27T12:42:56.510 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-06-27T12:42:56.646 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:56.637+0000 7fbc9c830700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:56.649 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:56.645+0000 7fbc9c830700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:56.709 INFO:tasks.rados.rados.0.smithi138.stdout:1144: finishing write tid 3 to smithi138138488-9 2023-06-27T12:42:56.709 INFO:tasks.rados.rados.0.smithi138.stdout:1152: seq_num 557 ranges {561499=407462,1527882=487521,2729901=557217} 2023-06-27T12:42:56.714 INFO:tasks.rados.rados.0.smithi138.stdout:1152: writing smithi138138488-19 from 561499 to 968961 tid 1 2023-06-27T12:42:56.720 INFO:tasks.rados.rados.0.smithi138.stdout:1152: writing smithi138138488-19 from 1527882 to 2015403 tid 2 2023-06-27T12:42:56.730 INFO:tasks.rados.rados.0.smithi138.stdout:1152: writing smithi138138488-19 from 2729901 to 3287118 tid 3 2023-06-27T12:42:56.730 INFO:tasks.rados.rados.0.smithi138.stdout:1143: done (4 left) 2023-06-27T12:42:56.730 INFO:tasks.rados.rados.0.smithi138.stdout:1153: write (excl) oid 20 current snap is 132 2023-06-27T12:42:56.730 INFO:tasks.rados.rados.0.smithi138.stdout:1145: expect (ObjNum 547 snap 128 seq_num 547) 2023-06-27T12:42:56.861 INFO:tasks.rados.rados.0.smithi138.stdout:1144: finishing write tid 4 to smithi138138488-9 2023-06-27T12:42:56.862 INFO:tasks.rados.rados.0.smithi138.stdout:1144: finishing write tid 5 to smithi138138488-9 2023-06-27T12:42:56.862 INFO:tasks.rados.rados.0.smithi138.stdout:1144: oid 9 updating version 0 to 327 2023-06-27T12:42:56.862 INFO:tasks.rados.rados.0.smithi138.stdout:1144: oid 9 version 327 is already newer than 326 2023-06-27T12:42:56.862 INFO:tasks.rados.rados.0.smithi138.stdout:1144: oid 9 version 327 is already newer than 327 2023-06-27T12:42:56.862 INFO:tasks.rados.rados.0.smithi138.stdout:1144: oid 9 version 327 is already newer than 327 2023-06-27T12:42:56.862 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 327 (ObjNum 552 snap 132 seq_num 552) dirty exists 2023-06-27T12:42:56.862 INFO:tasks.rados.rados.0.smithi138.stdout:1144: left oid 9 (ObjNum 552 snap 132 seq_num 552) 2023-06-27T12:42:56.863 INFO:tasks.rados.rados.0.smithi138.stdout:1153: seq_num 558 ranges {578306=482711,1586364=631179,2757324=484842} 2023-06-27T12:42:56.867 INFO:tasks.rados.rados.0.smithi138.stdout:1153: writing smithi138138488-20 from 578306 to 1061017 tid 1 2023-06-27T12:42:56.873 INFO:tasks.rados.rados.0.smithi138.stdout:1153: writing smithi138138488-20 from 1586364 to 2217543 tid 2 2023-06-27T12:42:56.879 INFO:tasks.rados.rados.0.smithi138.stdout:1153: writing smithi138138488-20 from 2757324 to 3242166 tid 3 2023-06-27T12:42:56.880 INFO:tasks.rados.rados.0.smithi138.stdout:1144: done (4 left) 2023-06-27T12:42:56.880 INFO:tasks.rados.rados.0.smithi138.stdout:1145: done (3 left) 2023-06-27T12:42:56.880 INFO:tasks.rados.rados.0.smithi138.stdout:1154: write oid 10 current snap is 132 2023-06-27T12:42:56.880 INFO:tasks.rados.rados.0.smithi138.stdout:1154: seq_num 559 ranges {595113=557960,1644846=491190,2784747=610366} 2023-06-27T12:42:56.885 INFO:tasks.rados.rados.0.smithi138.stdout:1154: writing smithi138138488-10 from 595113 to 1153073 tid 1 2023-06-27T12:42:56.891 INFO:tasks.rados.rados.0.smithi138.stdout:1154: writing smithi138138488-10 from 1644846 to 2136036 tid 2 2023-06-27T12:42:56.898 INFO:tasks.rados.rados.0.smithi138.stdout:1154: writing smithi138138488-10 from 2784747 to 3395113 tid 3 2023-06-27T12:42:56.898 INFO:tasks.rados.rados.0.smithi138.stdout:1155: write oid 9 current snap is 132 2023-06-27T12:42:56.898 INFO:tasks.rados.rados.0.smithi138.stdout:1147: expect (ObjNum 544 snap 128 seq_num 544) 2023-06-27T12:42:57.114 INFO:tasks.rados.rados.0.smithi138.stdout:1152: finishing write tid 1 to smithi138138488-19 2023-06-27T12:42:57.114 INFO:tasks.rados.rados.0.smithi138.stdout:1155: seq_num 560 ranges {611920=633209,1703328=634848,2812170=507386} 2023-06-27T12:42:57.120 INFO:tasks.rados.rados.0.smithi138.stdout:1155: writing smithi138138488-9 from 611920 to 1245129 tid 1 2023-06-27T12:42:57.128 INFO:tasks.rados.rados.0.smithi138.stdout:1155: writing smithi138138488-9 from 1703328 to 2338176 tid 2 2023-06-27T12:42:57.134 INFO:tasks.rados.rados.0.smithi138.stdout:1155: writing smithi138138488-9 from 2812170 to 3319556 tid 3 2023-06-27T12:42:57.134 INFO:tasks.rados.rados.0.smithi138.stdout:1147: done (4 left) 2023-06-27T12:42:57.135 INFO:tasks.rados.rados.0.smithi138.stdout:1156: read oid 33 snap -1 2023-06-27T12:42:57.135 INFO:tasks.rados.rados.0.smithi138.stdout:1156: expect (ObjNum 450 snap 104 seq_num 450) 2023-06-27T12:42:57.135 INFO:tasks.rados.rados.0.smithi138.stdout:1157: write (excl) oid 22 current snap is 132 2023-06-27T12:42:57.135 INFO:tasks.rados.rados.0.smithi138.stdout:1152: finishing write tid 2 to smithi138138488-19 2023-06-27T12:42:57.135 INFO:tasks.rados.rados.0.smithi138.stdout:1152: finishing write tid 3 to smithi138138488-19 2023-06-27T12:42:57.135 INFO:tasks.rados.rados.0.smithi138.stdout:1152: finishing write tid 4 to smithi138138488-19 2023-06-27T12:42:57.135 INFO:tasks.rados.rados.0.smithi138.stdout:1152: finishing write tid 5 to smithi138138488-19 2023-06-27T12:42:57.136 INFO:tasks.rados.rados.0.smithi138.stdout:1152: oid 19 updating version 0 to 333 2023-06-27T12:42:57.136 INFO:tasks.rados.rados.0.smithi138.stdout:1152: oid 19 version 333 is already newer than 333 2023-06-27T12:42:57.136 INFO:tasks.rados.rados.0.smithi138.stdout:1152: oid 19 version 333 is already newer than 333 2023-06-27T12:42:57.136 INFO:tasks.rados.rados.0.smithi138.stdout:1152: oid 19 version 333 is already newer than 330 2023-06-27T12:42:57.136 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 333 (ObjNum 557 snap 132 seq_num 557) dirty exists 2023-06-27T12:42:57.136 INFO:tasks.rados.rados.0.smithi138.stdout:1152: left oid 19 (ObjNum 557 snap 132 seq_num 557) 2023-06-27T12:42:57.136 INFO:tasks.rados.rados.0.smithi138.stdout:1153: finishing write tid 1 to smithi138138488-20 2023-06-27T12:42:57.136 INFO:tasks.rados.rados.0.smithi138.stdout:1157: seq_num 561 ranges {628727=708458,2045457=494859,3123240=305487} 2023-06-27T12:42:57.141 INFO:tasks.rados.rados.0.smithi138.stdout:1157: writing smithi138138488-22 from 628727 to 1337185 tid 1 2023-06-27T12:42:57.148 INFO:tasks.rados.rados.0.smithi138.stdout:1157: writing smithi138138488-22 from 2045457 to 2540316 tid 2 2023-06-27T12:42:57.154 INFO:tasks.rados.rados.0.smithi138.stdout:1157: writing smithi138138488-22 from 3123240 to 3428727 tid 3 2023-06-27T12:42:57.155 INFO:tasks.rados.rados.0.smithi138.stdout:1152: done (5 left) 2023-06-27T12:42:57.155 INFO:tasks.rados.rados.0.smithi138.stdout:1158: read oid 15 snap -1 2023-06-27T12:42:57.155 INFO:tasks.rados.rados.0.smithi138.stdout:1158: expect (ObjNum 553 snap 132 seq_num 553) 2023-06-27T12:42:57.155 INFO:tasks.rados.rados.0.smithi138.stdout:1159: write oid 16 current snap is 132 2023-06-27T12:42:57.155 INFO:tasks.rados.rados.0.smithi138.stdout:1159: seq_num 562 ranges {645534=783707,2103939=638517,3150663=294871} 2023-06-27T12:42:57.163 INFO:tasks.rados.rados.0.smithi138.stdout:1159: writing smithi138138488-16 from 645534 to 1429241 tid 1 2023-06-27T12:42:57.172 INFO:tasks.rados.rados.0.smithi138.stdout:1159: writing smithi138138488-16 from 2103939 to 2742456 tid 2 2023-06-27T12:42:57.177 INFO:tasks.rados.rados.0.smithi138.stdout:1159: writing smithi138138488-16 from 3150663 to 3445534 tid 3 2023-06-27T12:42:57.177 INFO:tasks.rados.rados.0.smithi138.stdout:1160: read oid 45 snap 130 2023-06-27T12:42:57.177 INFO:tasks.rados.rados.0.smithi138.stdout:1160: expect (ObjNum 513 snap 117 seq_num 513) 2023-06-27T12:42:57.178 INFO:tasks.rados.rados.0.smithi138.stdout:1161: snap_remove snap 131 2023-06-27T12:42:57.490 INFO:tasks.rados.rados.0.smithi138.stdout:1161: done (8 left) 2023-06-27T12:42:57.490 INFO:tasks.rados.rados.0.smithi138.stdout:1162: write oid 31 current snap is 132 2023-06-27T12:42:57.490 INFO:tasks.rados.rados.0.smithi138.stdout:1154: finishing write tid 1 to smithi138138488-10 2023-06-27T12:42:57.490 INFO:tasks.rados.rados.0.smithi138.stdout:1162: seq_num 563 ranges {662341=575309,1878774=498528,2894439=457555} 2023-06-27T12:42:57.492 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.2 to 1 (10000) 2023-06-27T12:42:57.499 INFO:tasks.rados.rados.0.smithi138.stdout:1162: writing smithi138138488-31 from 662341 to 1237650 tid 1 2023-06-27T12:42:57.507 INFO:tasks.rados.rados.0.smithi138.stdout:1162: writing smithi138138488-31 from 1878774 to 2377302 tid 2 2023-06-27T12:42:57.508 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-06-27T12:42:57.515 INFO:tasks.rados.rados.0.smithi138.stdout:1162: writing smithi138138488-31 from 2894439 to 3351994 tid 3 2023-06-27T12:42:57.515 INFO:tasks.rados.rados.0.smithi138.stdout:1163: read oid 40 snap -1 2023-06-27T12:42:57.515 INFO:tasks.rados.rados.0.smithi138.stdout:1163: expect (ObjNum 536 snap 124 seq_num 536) 2023-06-27T12:42:57.516 INFO:tasks.rados.rados.0.smithi138.stdout:1164: delete oid 34 current snap is 132 2023-06-27T12:42:57.516 INFO:tasks.rados.rados.0.smithi138.stdout:1154: finishing write tid 2 to smithi138138488-10 2023-06-27T12:42:57.516 INFO:tasks.rados.rados.0.smithi138.stdout:1154: finishing write tid 3 to smithi138138488-10 2023-06-27T12:42:57.516 INFO:tasks.rados.rados.0.smithi138.stdout:1154: finishing write tid 4 to smithi138138488-10 2023-06-27T12:42:57.516 INFO:tasks.rados.rados.0.smithi138.stdout:1154: finishing write tid 5 to smithi138138488-10 2023-06-27T12:42:57.516 INFO:tasks.rados.rados.0.smithi138.stdout:1154: oid 10 updating version 0 to 159 2023-06-27T12:42:57.516 INFO:tasks.rados.rados.0.smithi138.stdout:1154: oid 10 version 159 is already newer than 159 2023-06-27T12:42:57.517 INFO:tasks.rados.rados.0.smithi138.stdout:1154: oid 10 version 159 is already newer than 159 2023-06-27T12:42:57.517 INFO:tasks.rados.rados.0.smithi138.stdout:1154: oid 10 version 159 is already newer than 159 2023-06-27T12:42:57.517 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 159 (ObjNum 559 snap 132 seq_num 559) dirty exists 2023-06-27T12:42:57.517 INFO:tasks.rados.rados.0.smithi138.stdout:1154: left oid 10 (ObjNum 559 snap 132 seq_num 559) 2023-06-27T12:42:57.517 INFO:tasks.rados.rados.0.smithi138.stdout:1153: finishing write tid 2 to smithi138138488-20 2023-06-27T12:42:57.517 INFO:tasks.rados.rados.0.smithi138.stdout:1153: finishing write tid 3 to smithi138138488-20 2023-06-27T12:42:57.517 INFO:tasks.rados.rados.0.smithi138.stdout:1153: finishing write tid 4 to smithi138138488-20 2023-06-27T12:42:57.517 INFO:tasks.rados.rados.0.smithi138.stdout:1153: finishing write tid 5 to smithi138138488-20 2023-06-27T12:42:57.518 INFO:tasks.rados.rados.0.smithi138.stdout:1153: oid 20 updating version 0 to 170 2023-06-27T12:42:57.518 INFO:tasks.rados.rados.0.smithi138.stdout:1153: oid 20 version 170 is already newer than 169 2023-06-27T12:42:57.518 INFO:tasks.rados.rados.0.smithi138.stdout:1153: oid 20 updating version 170 to 174 2023-06-27T12:42:57.518 INFO:tasks.rados.rados.0.smithi138.stdout:1153: oid 20 version 174 is already newer than 174 2023-06-27T12:42:57.518 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 174 (ObjNum 558 snap 132 seq_num 558) dirty exists 2023-06-27T12:42:57.518 INFO:tasks.rados.rados.0.smithi138.stdout:1153: left oid 20 (ObjNum 558 snap 132 seq_num 558) 2023-06-27T12:42:57.518 INFO:tasks.rados.rados.0.smithi138.stdout:1155: finishing write tid 1 to smithi138138488-9 2023-06-27T12:42:57.518 INFO:tasks.rados.rados.0.smithi138.stdout:1156: expect (ObjNum 450 snap 104 seq_num 450) 2023-06-27T12:42:57.659 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:57.650+0000 7f96c0c6c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:57.662 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:57.658+0000 7f96c0c6c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:57.861 INFO:tasks.rados.rados.0.smithi138.stdout:1155: finishing write tid 2 to smithi138138488-9 2023-06-27T12:42:57.862 INFO:tasks.rados.rados.0.smithi138.stdout:1160: expect (ObjNum 513 snap 117 seq_num 513) 2023-06-27T12:42:57.927 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:42:58.000 INFO:tasks.rados.rados.0.smithi138.stdout:1157: finishing write tid 1 to smithi138138488-22 2023-06-27T12:42:58.000 INFO:tasks.rados.rados.0.smithi138.stdout:1158: expect (ObjNum 553 snap 132 seq_num 553) 2023-06-27T12:42:58.054 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:58.049+0000 7f37e2ac6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:58.057 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:58.053+0000 7f37e2ac6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:42:58.104 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:42:58.105 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:42:58.128 INFO:tasks.rados.rados.0.smithi138.stdout:1155: finishing write tid 3 to smithi138138488-9 2023-06-27T12:42:58.128 INFO:tasks.rados.rados.0.smithi138.stdout:1157: finishing write tid 2 to smithi138138488-22 2023-06-27T12:42:58.128 INFO:tasks.rados.rados.0.smithi138.stdout:1157: finishing write tid 3 to smithi138138488-22 2023-06-27T12:42:58.129 INFO:tasks.rados.rados.0.smithi138.stdout:1157: finishing write tid 4 to smithi138138488-22 2023-06-27T12:42:58.129 INFO:tasks.rados.rados.0.smithi138.stdout:1157: finishing write tid 5 to smithi138138488-22 2023-06-27T12:42:58.129 INFO:tasks.rados.rados.0.smithi138.stdout:1157: oid 22 updating version 0 to 207 2023-06-27T12:42:58.129 INFO:tasks.rados.rados.0.smithi138.stdout:1157: oid 22 version 207 is already newer than 207 2023-06-27T12:42:58.129 INFO:tasks.rados.rados.0.smithi138.stdout:1157: oid 22 version 207 is already newer than 203 2023-06-27T12:42:58.129 INFO:tasks.rados.rados.0.smithi138.stdout:1157: oid 22 version 207 is already newer than 207 2023-06-27T12:42:58.129 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 207 (ObjNum 561 snap 132 seq_num 561) dirty exists 2023-06-27T12:42:58.129 INFO:tasks.rados.rados.0.smithi138.stdout:1157: left oid 22 (ObjNum 561 snap 132 seq_num 561) 2023-06-27T12:42:58.130 INFO:tasks.rados.rados.0.smithi138.stdout:1159: finishing write tid 1 to smithi138138488-16 2023-06-27T12:42:58.130 INFO:tasks.rados.rados.0.smithi138.stdout:1155: finishing write tid 4 to smithi138138488-9 2023-06-27T12:42:58.130 INFO:tasks.rados.rados.0.smithi138.stdout:1155: finishing write tid 5 to smithi138138488-9 2023-06-27T12:42:58.130 INFO:tasks.rados.rados.0.smithi138.stdout:1155: oid 9 updating version 0 to 334 2023-06-27T12:42:58.130 INFO:tasks.rados.rados.0.smithi138.stdout:1155: oid 9 updating version 334 to 337 2023-06-27T12:42:58.130 INFO:tasks.rados.rados.0.smithi138.stdout:1155: oid 9 version 337 is already newer than 337 2023-06-27T12:42:58.130 INFO:tasks.rados.rados.0.smithi138.stdout:1155: oid 9 version 337 is already newer than 337 2023-06-27T12:42:58.130 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 337 (ObjNum 560 snap 132 seq_num 560) dirty exists 2023-06-27T12:42:58.131 INFO:tasks.rados.rados.0.smithi138.stdout:1155: left oid 9 (ObjNum 560 snap 132 seq_num 560) 2023-06-27T12:42:58.131 INFO:tasks.rados.rados.0.smithi138.stdout:1159: finishing write tid 2 to smithi138138488-16 2023-06-27T12:42:58.131 INFO:tasks.rados.rados.0.smithi138.stdout:1159: finishing write tid 3 to smithi138138488-16 2023-06-27T12:42:58.131 INFO:tasks.rados.rados.0.smithi138.stdout:1159: finishing write tid 4 to smithi138138488-16 2023-06-27T12:42:58.131 INFO:tasks.rados.rados.0.smithi138.stdout:1153: done (10 left) 2023-06-27T12:42:58.131 INFO:tasks.rados.rados.0.smithi138.stdout:1154: done (9 left) 2023-06-27T12:42:58.131 INFO:tasks.rados.rados.0.smithi138.stdout:1155: done (8 left) 2023-06-27T12:42:58.131 INFO:tasks.rados.rados.0.smithi138.stdout:1156: done (7 left) 2023-06-27T12:42:58.132 INFO:tasks.rados.rados.0.smithi138.stdout:1157: done (6 left) 2023-06-27T12:42:58.132 INFO:tasks.rados.rados.0.smithi138.stdout:1158: done (5 left) 2023-06-27T12:42:58.132 INFO:tasks.rados.rados.0.smithi138.stdout:1160: done (4 left) 2023-06-27T12:42:58.132 INFO:tasks.rados.rados.0.smithi138.stdout:1164: done (3 left) 2023-06-27T12:42:58.132 INFO:tasks.rados.rados.0.smithi138.stdout:1165: read oid 11 snap -1 2023-06-27T12:42:58.132 INFO:tasks.rados.rados.0.smithi138.stdout:1165: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:42:58.132 INFO:tasks.rados.rados.0.smithi138.stdout:1166: snap_remove snap 113 2023-06-27T12:42:58.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~143s 2023-06-27T12:42:58.508 INFO:tasks.rados.rados.0.smithi138.stdout:1166: done (4 left) 2023-06-27T12:42:58.509 INFO:tasks.rados.rados.0.smithi138.stdout:1167: snap_create 2023-06-27T12:42:58.509 INFO:tasks.rados.rados.0.smithi138.stdout:1159: finishing write tid 5 to smithi138138488-16 2023-06-27T12:42:58.509 INFO:tasks.rados.rados.0.smithi138.stdout:1159: oid 16 updating version 0 to 355 2023-06-27T12:42:58.509 INFO:tasks.rados.rados.0.smithi138.stdout:1159: oid 16 version 355 is already newer than 355 2023-06-27T12:42:58.509 INFO:tasks.rados.rados.0.smithi138.stdout:1159: oid 16 version 355 is already newer than 355 2023-06-27T12:42:58.510 INFO:tasks.rados.rados.0.smithi138.stdout:1159: oid 16 version 355 is already newer than 355 2023-06-27T12:42:58.510 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 355 (ObjNum 562 snap 132 seq_num 562) dirty exists 2023-06-27T12:42:58.510 INFO:tasks.rados.rados.0.smithi138.stdout:1159: left oid 16 (ObjNum 562 snap 132 seq_num 562) 2023-06-27T12:42:58.510 INFO:tasks.rados.rados.0.smithi138.stdout:1162: finishing write tid 1 to smithi138138488-31 2023-06-27T12:42:58.510 INFO:tasks.rados.rados.0.smithi138.stdout:1162: finishing write tid 2 to smithi138138488-31 2023-06-27T12:42:58.510 INFO:tasks.rados.rados.0.smithi138.stdout:1163: expect (ObjNum 536 snap 124 seq_num 536) 2023-06-27T12:42:58.511 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.3 to 1 (10000) 2023-06-27T12:42:58.523 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-06-27T12:42:58.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:58.667 INFO:tasks.rados.rados.0.smithi138.stdout:1162: finishing write tid 3 to smithi138138488-31 2023-06-27T12:42:58.667 INFO:tasks.rados.rados.0.smithi138.stdout:1162: finishing write tid 4 to smithi138138488-31 2023-06-27T12:42:58.667 INFO:tasks.rados.rados.0.smithi138.stdout:1162: finishing write tid 5 to smithi138138488-31 2023-06-27T12:42:58.667 INFO:tasks.rados.rados.0.smithi138.stdout:1162: oid 31 updating version 0 to 362 2023-06-27T12:42:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1162: oid 31 version 362 is already newer than 362 2023-06-27T12:42:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1162: oid 31 version 362 is already newer than 362 2023-06-27T12:42:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1162: oid 31 version 362 is already newer than 362 2023-06-27T12:42:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 362 (ObjNum 563 snap 132 seq_num 563) dirty exists 2023-06-27T12:42:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1162: left oid 31 (ObjNum 563 snap 132 seq_num 563) 2023-06-27T12:42:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1165: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:42:58.693 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:58.686+0000 7fa6d2bcd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:58.696 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:58.692+0000 7fa6d2bcd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:58.921 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:58.923 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2023-06-27T12:42:58.503789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":102324217,"num_objects":274,"num_object_clones":228,"num_object_copies":822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1651,"num_read_kb":1179970,"num_write":1469,"num_write_kb":425176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2750,"ondisk_log_size":2750,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":853524,"kb_used_data":717556,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376633836,"statfs":{"total":386547056640,"available":385673048064,"internally_reserved":0,"allocated":734777344,"data_stored":824202066,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027929"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":669,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.497510+0000","last_change":"2023-06-27T12:42:50.486943+0000","last_active":"2023-06-27T12:42:53.497510+0000","last_peered":"2023-06-27T12:42:53.497510+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:53.497510+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"262'161","reported_seq":873,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.970558+0000","last_change":"2023-06-27T12:42:50.509501+0000","last_active":"2023-06-27T12:42:53.970558+0000","last_peered":"2023-06-27T12:42:53.970558+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:53.970558+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9428958,"num_objects":21,"num_object_clones":17,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":122,"num_read_kb":147869,"num_write":114,"num_write_kb":33537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"279'231","reported_seq":876,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.945671+0000","last_change":"2023-06-27T12:42:51.678601+0000","last_active":"2023-06-27T12:42:53.945671+0000","last_peered":"2023-06-27T12:42:53.945671+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:53.945671+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15109955,"num_objects":26,"num_object_clones":21,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":175,"num_read_kb":100322,"num_write":160,"num_write_kb":45965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":26}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"268'111","reported_seq":803,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.486900+0000","last_change":"2023-06-27T12:42:50.533312+0000","last_active":"2023-06-27T12:42:53.486900+0000","last_peered":"2023-06-27T12:42:53.486900+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:53.486900+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9529833,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":100,"num_read_kb":54528,"num_write":88,"num_write_kb":24907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"263'73","reported_seq":368,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.494359+0000","last_change":"2023-06-27T12:42:50.474466+0000","last_active":"2023-06-27T12:42:53.494359+0000","last_peered":"2023-06-27T12:42:53.494359+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:53.494359+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2907710,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":69,"num_read_kb":34136,"num_write":34,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"280'346","reported_seq":900,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.537658+0000","last_change":"2023-06-27T12:42:50.475126+0000","last_active":"2023-06-27T12:42:53.537658+0000","last_peered":"2023-06-27T12:42:53.537658+0000","last_clean":"2023-06-27T12:42:53.537658+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:53.537658+0000","last_undegraded":"2023-06-27T12:42:53.537658+0000","last_fullsized":"2023-06-27T12:42:53.537658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630972,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":274,"num_read_kb":171286,"num_write":210,"num_write_kb":60801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"249'154","reported_seq":713,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.497490+0000","last_change":"2023-06-27T12:42:50.492591+0000","last_active":"2023-06-27T12:42:53.497490+0000","last_peered":"2023-06-27T12:42:53.497490+0000","last_clean":"2023-06-27T12:42:53.497490+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:53.497490+0000","last_undegraded":"2023-06-27T12:42:53.497490+0000","last_fullsized":"2023-06-27T12:42:53.497490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":110,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":711,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.944676+0000","last_change":"2023-06-27T12:42:50.520775+0000","last_active":"2023-06-27T12:42:53.944676+0000","last_peered":"2023-06-27T12:42:53.944676+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:53.944676+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":464,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.964360+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:53.964360+0000","last_peered":"2023-06-27T12:42:53.964360+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:53.964360+0000","last_undegraded":"2023-06-27T12:42:53.964360+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":625,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.948248+0000","last_change":"2023-06-27T12:42:50.526800+0000","last_active":"2023-06-27T12:42:53.948248+0000","last_peered":"2023-06-27T12:42:53.948248+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:53.948248+0000","last_undegraded":"2023-06-27T12:42:53.948248+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":443,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.489347+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:53.489347+0000","last_peered":"2023-06-27T12:42:53.489347+0000","last_clean":"2023-06-27T12:42:53.489347+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:53.489347+0000","last_undegraded":"2023-06-27T12:42:53.489347+0000","last_fullsized":"2023-06-27T12:42:53.489347+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"261'150","reported_seq":811,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.965281+0000","last_change":"2023-06-27T12:42:50.510594+0000","last_active":"2023-06-27T12:42:53.965281+0000","last_peered":"2023-06-27T12:42:53.965281+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:53.965281+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563315,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":66415,"num_write":101,"num_write_kb":27991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":434,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.947372+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:53.947372+0000","last_peered":"2023-06-27T12:42:53.947372+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:53.947372+0000","last_undegraded":"2023-06-27T12:42:53.947372+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"268'343","reported_seq":443,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.972516+0000","last_change":"2023-06-27T12:42:50.511614+0000","last_active":"2023-06-27T12:42:53.972516+0000","last_peered":"2023-06-27T12:42:53.972516+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:53.972516+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6487733,"num_objects":12,"num_object_clones":11,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":41,"num_read_kb":24503,"num_write":68,"num_write_kb":20973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":431,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.942666+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:53.942666+0000","last_peered":"2023-06-27T12:42:53.942666+0000","last_clean":"2023-06-27T12:42:53.942666+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:53.942666+0000","last_undegraded":"2023-06-27T12:42:53.942666+0000","last_fullsized":"2023-06-27T12:42:53.942666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":195,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.490513+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:53.490513+0000","last_peered":"2023-06-27T12:42:53.490513+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:53.490513+0000","last_undegraded":"2023-06-27T12:42:53.490513+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"273'316","reported_seq":996,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.943328+0000","last_change":"2023-06-27T12:42:51.603912+0000","last_active":"2023-06-27T12:42:53.943328+0000","last_peered":"2023-06-27T12:42:53.943328+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:53.943328+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8386990,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":278,"num_read_kb":173896,"num_write":232,"num_write_kb":67126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":443,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.489851+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:53.489851+0000","last_peered":"2023-06-27T12:42:53.489851+0000","last_clean":"2023-06-27T12:42:53.489851+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:53.489851+0000","last_undegraded":"2023-06-27T12:42:53.489851+0000","last_fullsized":"2023-06-27T12:42:53.489851+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"279'189","reported_seq":391,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.944097+0000","last_change":"2023-06-27T12:42:51.606477+0000","last_active":"2023-06-27T12:42:53.944097+0000","last_peered":"2023-06-27T12:42:53.944097+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:53.944097+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3227043,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":41527,"num_write":54,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":207,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.969894+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:53.969894+0000","last_peered":"2023-06-27T12:42:53.969894+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:53.969894+0000","last_undegraded":"2023-06-27T12:42:53.969894+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"269'169","reported_seq":844,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.972575+0000","last_change":"2023-06-27T12:42:50.527105+0000","last_active":"2023-06-27T12:42:53.972575+0000","last_peered":"2023-06-27T12:42:53.972575+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:53.972575+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714710,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":93,"num_read_kb":105200,"num_write":114,"num_write_kb":33633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":378,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.948010+0000","last_change":"2023-06-27T12:42:50.513033+0000","last_active":"2023-06-27T12:42:53.948010+0000","last_peered":"2023-06-27T12:42:53.948010+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:53.948010+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":42747,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":464,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.972915+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:53.972915+0000","last_peered":"2023-06-27T12:42:53.972915+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:53.972915+0000","last_undegraded":"2023-06-27T12:42:53.972915+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":431,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.942041+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:53.942041+0000","last_peered":"2023-06-27T12:42:53.942041+0000","last_clean":"2023-06-27T12:42:53.942041+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:53.942041+0000","last_undegraded":"2023-06-27T12:42:53.942041+0000","last_fullsized":"2023-06-27T12:42:53.942041+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"277'202","reported_seq":842,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.971107+0000","last_change":"2023-06-27T12:42:50.499076+0000","last_active":"2023-06-27T12:42:53.971107+0000","last_peered":"2023-06-27T12:42:53.971107+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:53.971107+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6336437,"num_objects":22,"num_object_clones":19,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":96,"num_read_kb":92053,"num_write":106,"num_write_kb":30789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":102324217,"num_objects":274,"num_object_clones":228,"num_object_copies":822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1651,"num_read_kb":1179970,"num_write":1469,"num_write_kb":425176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2653,"ondisk_log_size":2653,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":209084,"kb_used_data":174068,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94162756,"statfs":{"total":96636764160,"available":96422662144,"internally_reserved":0,"allocated":178245632,"data_stored":199207065,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":222944,"kb_used_data":187928,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94148896,"statfs":{"total":96636764160,"available":96408469504,"internally_reserved":0,"allocated":192438272,"data_stored":213524294,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":240216,"kb_used_data":205200,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94131624,"statfs":{"total":96636764160,"available":96390782976,"internally_reserved":0,"allocated":210124800,"data_stored":234431155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:58.937 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:59.063 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:59.056+0000 7f5bd5a00700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:59.066 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:59.063+0000 7f5bcffff700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:59.264 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:59.275 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2023-06-27T12:42:58.503789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":102324217,"num_objects":274,"num_object_clones":228,"num_object_copies":822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1651,"num_read_kb":1179970,"num_write":1469,"num_write_kb":425176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2750,"ondisk_log_size":2750,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":853524,"kb_used_data":717556,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376633836,"statfs":{"total":386547056640,"available":385673048064,"internally_reserved":0,"allocated":734777344,"data_stored":824202066,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027929"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":669,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.497510+0000","last_change":"2023-06-27T12:42:50.486943+0000","last_active":"2023-06-27T12:42:53.497510+0000","last_peered":"2023-06-27T12:42:53.497510+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:53.497510+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"262'161","reported_seq":873,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.970558+0000","last_change":"2023-06-27T12:42:50.509501+0000","last_active":"2023-06-27T12:42:53.970558+0000","last_peered":"2023-06-27T12:42:53.970558+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:53.970558+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9428958,"num_objects":21,"num_object_clones":17,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":122,"num_read_kb":147869,"num_write":114,"num_write_kb":33537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"279'231","reported_seq":876,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.945671+0000","last_change":"2023-06-27T12:42:51.678601+0000","last_active":"2023-06-27T12:42:53.945671+0000","last_peered":"2023-06-27T12:42:53.945671+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:53.945671+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15109955,"num_objects":26,"num_object_clones":21,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":175,"num_read_kb":100322,"num_write":160,"num_write_kb":45965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":26}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"268'111","reported_seq":803,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.486900+0000","last_change":"2023-06-27T12:42:50.533312+0000","last_active":"2023-06-27T12:42:53.486900+0000","last_peered":"2023-06-27T12:42:53.486900+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:53.486900+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9529833,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":100,"num_read_kb":54528,"num_write":88,"num_write_kb":24907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"263'73","reported_seq":368,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.494359+0000","last_change":"2023-06-27T12:42:50.474466+0000","last_active":"2023-06-27T12:42:53.494359+0000","last_peered":"2023-06-27T12:42:53.494359+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:53.494359+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2907710,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":69,"num_read_kb":34136,"num_write":34,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"280'346","reported_seq":900,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.537658+0000","last_change":"2023-06-27T12:42:50.475126+0000","last_active":"2023-06-27T12:42:53.537658+0000","last_peered":"2023-06-27T12:42:53.537658+0000","last_clean":"2023-06-27T12:42:53.537658+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:53.537658+0000","last_undegraded":"2023-06-27T12:42:53.537658+0000","last_fullsized":"2023-06-27T12:42:53.537658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630972,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":274,"num_read_kb":171286,"num_write":210,"num_write_kb":60801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"249'154","reported_seq":713,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.497490+0000","last_change":"2023-06-27T12:42:50.492591+0000","last_active":"2023-06-27T12:42:53.497490+0000","last_peered":"2023-06-27T12:42:53.497490+0000","last_clean":"2023-06-27T12:42:53.497490+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:53.497490+0000","last_undegraded":"2023-06-27T12:42:53.497490+0000","last_fullsized":"2023-06-27T12:42:53.497490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":110,"num_read_kb":78355,"num_write":85,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":711,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.944676+0000","last_change":"2023-06-27T12:42:50.520775+0000","last_active":"2023-06-27T12:42:53.944676+0000","last_peered":"2023-06-27T12:42:53.944676+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:53.944676+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":464,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.964360+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:53.964360+0000","last_peered":"2023-06-27T12:42:53.964360+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:53.964360+0000","last_undegraded":"2023-06-27T12:42:53.964360+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":625,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.948248+0000","last_change":"2023-06-27T12:42:50.526800+0000","last_active":"2023-06-27T12:42:53.948248+0000","last_peered":"2023-06-27T12:42:53.948248+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:53.948248+0000","last_undegraded":"2023-06-27T12:42:53.948248+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":443,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.489347+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:53.489347+0000","last_peered":"2023-06-27T12:42:53.489347+0000","last_clean":"2023-06-27T12:42:53.489347+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:53.489347+0000","last_undegraded":"2023-06-27T12:42:53.489347+0000","last_fullsized":"2023-06-27T12:42:53.489347+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"261'150","reported_seq":811,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.965281+0000","last_change":"2023-06-27T12:42:50.510594+0000","last_active":"2023-06-27T12:42:53.965281+0000","last_peered":"2023-06-27T12:42:53.965281+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:53.965281+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563315,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":66415,"num_write":101,"num_write_kb":27991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":434,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.947372+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:53.947372+0000","last_peered":"2023-06-27T12:42:53.947372+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:53.947372+0000","last_undegraded":"2023-06-27T12:42:53.947372+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"268'343","reported_seq":443,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.972516+0000","last_change":"2023-06-27T12:42:50.511614+0000","last_active":"2023-06-27T12:42:53.972516+0000","last_peered":"2023-06-27T12:42:53.972516+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:53.972516+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6487733,"num_objects":12,"num_object_clones":11,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":41,"num_read_kb":24503,"num_write":68,"num_write_kb":20973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":431,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.942666+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:53.942666+0000","last_peered":"2023-06-27T12:42:53.942666+0000","last_clean":"2023-06-27T12:42:53.942666+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:53.942666+0000","last_undegraded":"2023-06-27T12:42:53.942666+0000","last_fullsized":"2023-06-27T12:42:53.942666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":195,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.490513+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:53.490513+0000","last_peered":"2023-06-27T12:42:53.490513+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:53.490513+0000","last_undegraded":"2023-06-27T12:42:53.490513+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"273'316","reported_seq":996,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.943328+0000","last_change":"2023-06-27T12:42:51.603912+0000","last_active":"2023-06-27T12:42:53.943328+0000","last_peered":"2023-06-27T12:42:53.943328+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:53.943328+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8386990,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":278,"num_read_kb":173896,"num_write":232,"num_write_kb":67126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":443,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.489851+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:53.489851+0000","last_peered":"2023-06-27T12:42:53.489851+0000","last_clean":"2023-06-27T12:42:53.489851+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:53.489851+0000","last_undegraded":"2023-06-27T12:42:53.489851+0000","last_fullsized":"2023-06-27T12:42:53.489851+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"279'189","reported_seq":391,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.944097+0000","last_change":"2023-06-27T12:42:51.606477+0000","last_active":"2023-06-27T12:42:53.944097+0000","last_peered":"2023-06-27T12:42:53.944097+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:53.944097+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3227043,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":41527,"num_write":54,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":207,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.969894+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:53.969894+0000","last_peered":"2023-06-27T12:42:53.969894+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:53.969894+0000","last_undegraded":"2023-06-27T12:42:53.969894+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"269'169","reported_seq":844,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.972575+0000","last_change":"2023-06-27T12:42:50.527105+0000","last_active":"2023-06-27T12:42:53.972575+0000","last_peered":"2023-06-27T12:42:53.972575+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:53.972575+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714710,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":93,"num_read_kb":105200,"num_write":114,"num_write_kb":33633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":378,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.948010+0000","last_change":"2023-06-27T12:42:50.513033+0000","last_active":"2023-06-27T12:42:53.948010+0000","last_peered":"2023-06-27T12:42:53.948010+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:53.948010+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":42747,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":464,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-06-27T12:42:53.972915+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:53.972915+0000","last_peered":"2023-06-27T12:42:53.972915+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:53.972915+0000","last_undegraded":"2023-06-27T12:42:53.972915+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":431,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-06-27T12:42:53.942041+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:53.942041+0000","last_peered":"2023-06-27T12:42:53.942041+0000","last_clean":"2023-06-27T12:42:53.942041+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:53.942041+0000","last_undegraded":"2023-06-27T12:42:53.942041+0000","last_fullsized":"2023-06-27T12:42:53.942041+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"277'202","reported_seq":842,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:53.971107+0000","last_change":"2023-06-27T12:42:50.499076+0000","last_active":"2023-06-27T12:42:53.971107+0000","last_peered":"2023-06-27T12:42:53.971107+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:53.971107+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6336437,"num_objects":22,"num_object_clones":19,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":96,"num_read_kb":92053,"num_write":106,"num_write_kb":30789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":102324217,"num_objects":274,"num_object_clones":228,"num_object_copies":822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1651,"num_read_kb":1179970,"num_write":1469,"num_write_kb":425176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2653,"ondisk_log_size":2653,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":209084,"kb_used_data":174068,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94162756,"statfs":{"total":96636764160,"available":96422662144,"internally_reserved":0,"allocated":178245632,"data_stored":199207065,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":222944,"kb_used_data":187928,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94148896,"statfs":{"total":96636764160,"available":96408469504,"internally_reserved":0,"allocated":192438272,"data_stored":213524294,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":240216,"kb_used_data":205200,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94131624,"statfs":{"total":96636764160,"available":96390782976,"internally_reserved":0,"allocated":210124800,"data_stored":234431155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:59.277 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-06-27T12:42:59.278 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:42:59.278 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:42:59.404 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:59.397+0000 7f85fc606700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:59.407 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:59.404+0000 7f85fc606700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:59.509 INFO:tasks.rados.rados.0.smithi138.stdout:1159: done (4 left) 2023-06-27T12:42:59.509 INFO:tasks.rados.rados.0.smithi138.stdout:1162: done (3 left) 2023-06-27T12:42:59.510 INFO:tasks.rados.rados.0.smithi138.stdout:1163: done (2 left) 2023-06-27T12:42:59.510 INFO:tasks.rados.rados.0.smithi138.stdout:1165: done (1 left) 2023-06-27T12:42:59.510 INFO:tasks.rados.rados.0.smithi138.stdout:1167: done (0 left) 2023-06-27T12:42:59.510 INFO:tasks.rados.rados.0.smithi138.stdout:1168: write oid 35 current snap is 133 2023-06-27T12:42:59.511 INFO:tasks.rados.rados.0.smithi138.stdout:1168: seq_num 565 ranges {695955=725807,1879385=502197,2949285=480099} 2023-06-27T12:42:59.517 INFO:tasks.rados.rados.0.smithi138.stdout:1168: writing smithi138138488-35 from 695955 to 1421762 tid 1 2023-06-27T12:42:59.522 INFO:tasks.rados.rados.0.smithi138.stdout:1168: writing smithi138138488-35 from 1879385 to 2381582 tid 2 2023-06-27T12:42:59.528 INFO:tasks.rados.rados.0.smithi138.stdout:1168: writing smithi138138488-35 from 2949285 to 3429384 tid 3 2023-06-27T12:42:59.528 INFO:tasks.rados.rados.0.smithi138.stdout:1169: read oid 33 snap 128 2023-06-27T12:42:59.529 INFO:tasks.rados.rados.0.smithi138.stdout:1169: expect (ObjNum 450 snap 104 seq_num 450) 2023-06-27T12:42:59.529 INFO:tasks.rados.rados.0.smithi138.stdout:1170: delete oid 26 current snap is 133 2023-06-27T12:42:59.559 INFO:tasks.rados.rados.0.smithi138.stdout:1168: finishing write tid 1 to smithi138138488-35 2023-06-27T12:42:59.563 INFO:tasks.rados.rados.0.smithi138.stdout:1168: finishing write tid 2 to smithi138138488-35 2023-06-27T12:42:59.589 INFO:tasks.rados.rados.0.smithi138.stdout:1170: done (2 left) 2023-06-27T12:42:59.589 INFO:tasks.rados.rados.0.smithi138.stdout:1171: read oid 27 snap -1 2023-06-27T12:42:59.589 INFO:tasks.rados.rados.0.smithi138.stdout:1171: expect (ObjNum 452 snap 104 seq_num 452) 2023-06-27T12:42:59.589 INFO:tasks.rados.rados.0.smithi138.stdout:1172: write oid 15 current snap is 133 2023-06-27T12:42:59.589 INFO:tasks.rados.rados.0.smithi138.stdout:1172: seq_num 567 ranges {729569=476305,1996349=505866,3004131=525438} 2023-06-27T12:42:59.597 INFO:tasks.rados.rados.0.smithi138.stdout:1172: writing smithi138138488-15 from 729569 to 1205874 tid 1 2023-06-27T12:42:59.607 INFO:tasks.rados.rados.0.smithi138.stdout:1172: writing smithi138138488-15 from 1996349 to 2502215 tid 2 2023-06-27T12:42:59.615 INFO:tasks.rados.rados.0.smithi138.stdout:1172: writing smithi138138488-15 from 3004131 to 3529569 tid 3 2023-06-27T12:42:59.615 INFO:tasks.rados.rados.0.smithi138.stdout:1173: write (excl) oid 45 current snap is 133 2023-06-27T12:42:59.615 INFO:tasks.rados.rados.0.smithi138.stdout:1173: seq_num 568 ranges {746376=551554,2054831=649524,3315201=231175} 2023-06-27T12:42:59.620 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:42:59.623 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:42:59.624 INFO:tasks.rados.rados.0.smithi138.stdout:1173: writing smithi138138488-45 from 746376 to 1297930 tid 1 2023-06-27T12:42:59.635 INFO:tasks.rados.rados.0.smithi138.stdout:1173: writing smithi138138488-45 from 2054831 to 2704355 tid 2 2023-06-27T12:42:59.636 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2023-06-27T12:42:59.504525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110038729,"num_objects":282,"num_object_clones":236,"num_object_copies":846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":1722,"num_read_kb":1239425,"num_write":1529,"num_write_kb":443605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2829,"ondisk_log_size":2829,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":877752,"kb_used_data":741784,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376609608,"statfs":{"total":386547056640,"available":385648238592,"internally_reserved":0,"allocated":759586816,"data_stored":854990704,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7714512,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":71,"num_read_kb":59455,"num_write":60,"num_write_kb":18429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.071937"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":682,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.851487+0000","last_change":"2023-06-27T12:42:58.861784+0000","last_active":"2023-06-27T12:42:58.851487+0000","last_peered":"2023-06-27T12:42:58.851487+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:42:58.851487+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"283'174","reported_seq":898,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.532732+0000","last_change":"2023-06-27T12:42:58.533255+0000","last_active":"2023-06-27T12:42:58.532732+0000","last_peered":"2023-06-27T12:42:58.532732+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:42:58.532732+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10756711,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":125,"num_read_kb":147871,"num_write":124,"num_write_kb":36349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"282'238","reported_seq":900,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.867996+0000","last_change":"2023-06-27T12:42:58.868771+0000","last_active":"2023-06-27T12:42:58.867996+0000","last_peered":"2023-06-27T12:42:58.867996+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:42:58.867996+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278025,"num_objects":27,"num_object_clones":22,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":113115,"num_write":165,"num_write_kb":47622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":27}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"268'111","reported_seq":822,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.862023+0000","last_change":"2023-06-27T12:42:58.862900+0000","last_active":"2023-06-27T12:42:58.862023+0000","last_peered":"2023-06-27T12:42:58.862023+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:42:58.862023+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9529833,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":114,"num_read_kb":71511,"num_write":88,"num_write_kb":24907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"282'77","reported_seq":389,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.852855+0000","last_change":"2023-06-27T12:42:58.853406+0000","last_active":"2023-06-27T12:42:58.852855+0000","last_peered":"2023-06-27T12:42:58.852855+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:42:58.852855+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6201981,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":40837,"num_write":39,"num_write_kb":11810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"284'363","reported_seq":926,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-06-27T12:42:58.842322+0000","last_change":"2023-06-27T12:42:58.910997+0000","last_active":"2023-06-27T12:42:58.842322+0000","last_peered":"2023-06-27T12:42:58.842322+0000","last_clean":"2023-06-27T12:42:58.842322+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:42:58.842322+0000","last_undegraded":"2023-06-27T12:42:58.842322+0000","last_fullsized":"2023-06-27T12:42:58.842322+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13748489,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":283,"num_read_kb":175221,"num_write":220,"num_write_kb":63975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"283'159","reported_seq":730,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-06-27T12:42:58.831452+0000","last_change":"2023-06-27T12:42:58.854961+0000","last_active":"2023-06-27T12:42:58.831452+0000","last_peered":"2023-06-27T12:42:58.831452+0000","last_clean":"2023-06-27T12:42:58.831452+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:42:58.831452+0000","last_undegraded":"2023-06-27T12:42:58.831452+0000","last_fullsized":"2023-06-27T12:42:58.831452+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395113,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":111,"num_read_kb":78356,"num_write":90,"num_write_kb":26241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":723,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.844448+0000","last_change":"2023-06-27T12:42:58.847307+0000","last_active":"2023-06-27T12:42:58.844448+0000","last_peered":"2023-06-27T12:42:58.844448+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:42:58.844448+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":474,"reported_epoch":285,"state":"active+undersized","last_fresh":"2023-06-27T12:42:58.521867+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:42:58.521867+0000","last_peered":"2023-06-27T12:42:58.521867+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:42:58.521867+0000","last_undegraded":"2023-06-27T12:42:58.521867+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":637,"reported_epoch":285,"state":"active+undersized","last_fresh":"2023-06-27T12:42:58.847497+0000","last_change":"2023-06-27T12:42:58.859410+0000","last_active":"2023-06-27T12:42:58.847497+0000","last_peered":"2023-06-27T12:42:58.847497+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:42:58.847497+0000","last_undegraded":"2023-06-27T12:42:58.847497+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":451,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-06-27T12:42:58.832277+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:42:58.832277+0000","last_peered":"2023-06-27T12:42:58.832277+0000","last_clean":"2023-06-27T12:42:58.832277+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:42:58.832277+0000","last_undegraded":"2023-06-27T12:42:58.832277+0000","last_fullsized":"2023-06-27T12:42:58.832277+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"261'150","reported_seq":826,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.531523+0000","last_change":"2023-06-27T12:42:58.532146+0000","last_active":"2023-06-27T12:42:58.531523+0000","last_peered":"2023-06-27T12:42:58.531523+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:42:58.531523+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563315,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":66415,"num_write":101,"num_write_kb":27991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":442,"reported_epoch":285,"state":"active+undersized","last_fresh":"2023-06-27T12:42:58.845059+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:42:58.845059+0000","last_peered":"2023-06-27T12:42:58.845059+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:42:58.845059+0000","last_undegraded":"2023-06-27T12:42:58.845059+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"282'350","reported_seq":466,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.534431+0000","last_change":"2023-06-27T12:42:58.534892+0000","last_active":"2023-06-27T12:42:58.534431+0000","last_peered":"2023-06-27T12:42:58.534431+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:42:58.534431+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3781707,"num_objects":14,"num_object_clones":13,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":29626,"num_write":73,"num_write_kb":22295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":14}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":439,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-06-27T12:42:58.846464+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:42:58.846464+0000","last_peered":"2023-06-27T12:42:58.846464+0000","last_clean":"2023-06-27T12:42:58.846464+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:42:58.846464+0000","last_undegraded":"2023-06-27T12:42:58.846464+0000","last_fullsized":"2023-06-27T12:42:58.846464+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":203,"reported_epoch":285,"state":"active+undersized","last_fresh":"2023-06-27T12:42:58.847802+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:42:58.847802+0000","last_peered":"2023-06-27T12:42:58.847802+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:42:58.847802+0000","last_undegraded":"2023-06-27T12:42:58.847802+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"283'337","reported_seq":1024,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.866327+0000","last_change":"2023-06-27T12:42:58.866988+0000","last_active":"2023-06-27T12:42:58.866327+0000","last_peered":"2023-06-27T12:42:58.866327+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:42:58.866327+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269506,"num_objects":43,"num_object_clones":37,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":282,"num_read_kb":173899,"num_write":247,"num_write_kb":71751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":43}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":451,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-06-27T12:42:58.845141+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:42:58.845141+0000","last_peered":"2023-06-27T12:42:58.845141+0000","last_clean":"2023-06-27T12:42:58.845141+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:42:58.845141+0000","last_undegraded":"2023-06-27T12:42:58.845141+0000","last_fullsized":"2023-06-27T12:42:58.845141+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"279'189","reported_seq":404,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.865553+0000","last_change":"2023-06-27T12:42:58.866186+0000","last_active":"2023-06-27T12:42:58.865553+0000","last_peered":"2023-06-27T12:42:58.865553+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:42:58.865553+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3227043,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":41527,"num_write":54,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":217,"reported_epoch":285,"state":"active+undersized","last_fresh":"2023-06-27T12:42:58.522237+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:42:58.522237+0000","last_peered":"2023-06-27T12:42:58.522237+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:42:58.522237+0000","last_undegraded":"2023-06-27T12:42:58.522237+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"269'169","reported_seq":859,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.530031+0000","last_change":"2023-06-27T12:42:58.530547+0000","last_active":"2023-06-27T12:42:58.530031+0000","last_peered":"2023-06-27T12:42:58.530031+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:42:58.530031+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714710,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":93,"num_read_kb":105200,"num_write":114,"num_write_kb":33633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":393,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.843505+0000","last_change":"2023-06-27T12:42:58.863457+0000","last_active":"2023-06-27T12:42:58.843505+0000","last_peered":"2023-06-27T12:42:58.843505+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:42:58.843505+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":48530,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":474,"reported_epoch":285,"state":"active+undersized","last_fresh":"2023-06-27T12:42:58.519727+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:42:58.519727+0000","last_peered":"2023-06-27T12:42:58.519727+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:42:58.519727+0000","last_undegraded":"2023-06-27T12:42:58.519727+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":439,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-06-27T12:42:58.839654+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:42:58.839654+0000","last_peered":"2023-06-27T12:42:58.839654+0000","last_clean":"2023-06-27T12:42:58.839654+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:42:58.839654+0000","last_undegraded":"2023-06-27T12:42:58.839654+0000","last_fullsized":"2023-06-27T12:42:58.839654+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"283'207","reported_seq":867,"reported_epoch":285,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:42:58.534379+0000","last_change":"2023-06-27T12:42:58.534833+0000","last_active":"2023-06-27T12:42:58.534379+0000","last_peered":"2023-06-27T12:42:58.534379+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:42:58.534379+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6571735,"num_objects":23,"num_object_clones":20,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":102,"num_read_kb":100184,"num_write":111,"num_write_kb":32264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":110038729,"num_objects":282,"num_object_clones":236,"num_object_copies":846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":1722,"num_read_kb":1239425,"num_write":1529,"num_write_kb":443605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2732,"ondisk_log_size":2732,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":218808,"kb_used_data":183792,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94153032,"statfs":{"total":96636764160,"available":96412704768,"internally_reserved":0,"allocated":188203008,"data_stored":211329061,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":228652,"kb_used_data":193636,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94143188,"statfs":{"total":96636764160,"available":96402624512,"internally_reserved":0,"allocated":198283264,"data_stored":220589113,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":249012,"kb_used_data":213996,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94122828,"statfs":{"total":96636764160,"available":96381775872,"internally_reserved":0,"allocated":219131904,"data_stored":246032978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:42:59.638 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.e 3.b 3.3 3.0 3.1 3.7 2023-06-27T12:42:59.641 INFO:tasks.rados.rados.0.smithi138.stdout:1173: writing smithi138138488-45 from 3315201 to 3546376 tid 3 2023-06-27T12:42:59.641 INFO:tasks.rados.rados.0.smithi138.stdout:1174: snap_remove snap 132 2023-06-27T12:42:59.776 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:59.763+0000 7f53036b8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:59.780 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:42:59.776+0000 7f53036b8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:42:59.973 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b] on osd.0 to force-backfill; instructing pg(s) [3.1,3.7] on osd.1 to force-backfill; instructing pg(s) [3.0,3.3,3.e] on osd.2 to force-backfill; 2023-06-27T12:42:59.973 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:00.106 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:00.231 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:00.226+0000 7f4a7ea08700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:00.234 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:00.231+0000 7f4a7ea08700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:00.279 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:00.281 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:00.282 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:00.515 INFO:tasks.rados.rados.0.smithi138.stdout:1174: done (5 left) 2023-06-27T12:43:00.516 INFO:tasks.rados.rados.0.smithi138.stdout:1175: delete oid 19 current snap is 133 2023-06-27T12:43:00.516 INFO:tasks.rados.rados.0.smithi138.stdout:1168: finishing write tid 3 to smithi138138488-35 2023-06-27T12:43:00.516 INFO:tasks.rados.rados.0.smithi138.stdout:1169: expect (ObjNum 450 snap 104 seq_num 450) 2023-06-27T12:43:00.853 INFO:tasks.rados.rados.0.smithi138.stdout:1168: finishing write tid 4 to smithi138138488-35 2023-06-27T12:43:00.853 INFO:tasks.rados.rados.0.smithi138.stdout:1168: finishing write tid 5 to smithi138138488-35 2023-06-27T12:43:00.853 INFO:tasks.rados.rados.0.smithi138.stdout:1168: oid 35 updating version 0 to 362 2023-06-27T12:43:00.854 INFO:tasks.rados.rados.0.smithi138.stdout:1168: oid 35 updating version 362 to 363 2023-06-27T12:43:00.854 INFO:tasks.rados.rados.0.smithi138.stdout:1168: oid 35 version 363 is already newer than 363 2023-06-27T12:43:00.854 INFO:tasks.rados.rados.0.smithi138.stdout:1168: oid 35 version 363 is already newer than 363 2023-06-27T12:43:00.854 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 363 (ObjNum 565 snap 133 seq_num 565) dirty exists 2023-06-27T12:43:00.854 INFO:tasks.rados.rados.0.smithi138.stdout:1168: left oid 35 (ObjNum 565 snap 133 seq_num 565) 2023-06-27T12:43:00.854 INFO:tasks.rados.rados.0.smithi138.stdout:1171: expect (ObjNum 452 snap 104 seq_num 452) 2023-06-27T12:43:00.989 INFO:tasks.rados.rados.0.smithi138.stdout:1172: finishing write tid 1 to smithi138138488-15 2023-06-27T12:43:00.990 INFO:tasks.rados.rados.0.smithi138.stdout:1172: finishing write tid 2 to smithi138138488-15 2023-06-27T12:43:00.990 INFO:tasks.rados.rados.0.smithi138.stdout:1173: finishing write tid 1 to smithi138138488-45 2023-06-27T12:43:00.990 INFO:tasks.rados.rados.0.smithi138.stdout:1172: finishing write tid 3 to smithi138138488-15 2023-06-27T12:43:00.990 INFO:tasks.rados.rados.0.smithi138.stdout:1172: finishing write tid 4 to smithi138138488-15 2023-06-27T12:43:00.990 INFO:tasks.rados.rados.0.smithi138.stdout:1172: finishing write tid 5 to smithi138138488-15 2023-06-27T12:43:00.990 INFO:tasks.rados.rados.0.smithi138.stdout:1172: oid 15 updating version 0 to 82 2023-06-27T12:43:00.990 INFO:tasks.rados.rados.0.smithi138.stdout:1172: oid 15 version 82 is already newer than 82 2023-06-27T12:43:00.991 INFO:tasks.rados.rados.0.smithi138.stdout:1172: oid 15 version 82 is already newer than 80 2023-06-27T12:43:00.991 INFO:tasks.rados.rados.0.smithi138.stdout:1172: oid 15 version 82 is already newer than 82 2023-06-27T12:43:00.991 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 82 (ObjNum 567 snap 133 seq_num 567) dirty exists 2023-06-27T12:43:00.991 INFO:tasks.rados.rados.0.smithi138.stdout:1172: left oid 15 (ObjNum 567 snap 133 seq_num 567) 2023-06-27T12:43:00.991 INFO:tasks.rados.rados.0.smithi138.stdout:1173: finishing write tid 2 to smithi138138488-45 2023-06-27T12:43:00.991 INFO:tasks.rados.rados.0.smithi138.stdout:1168: done (5 left) 2023-06-27T12:43:00.991 INFO:tasks.rados.rados.0.smithi138.stdout:1169: done (4 left) 2023-06-27T12:43:00.991 INFO:tasks.rados.rados.0.smithi138.stdout:1171: done (3 left) 2023-06-27T12:43:00.992 INFO:tasks.rados.rados.0.smithi138.stdout:1172: done (2 left) 2023-06-27T12:43:00.992 INFO:tasks.rados.rados.0.smithi138.stdout:1175: done (1 left) 2023-06-27T12:43:00.992 INFO:tasks.rados.rados.0.smithi138.stdout:1176: snap_create 2023-06-27T12:43:00.992 INFO:tasks.rados.rados.0.smithi138.stdout:1173: finishing write tid 3 to smithi138138488-45 2023-06-27T12:43:00.992 INFO:tasks.rados.rados.0.smithi138.stdout:1173: finishing write tid 4 to smithi138138488-45 2023-06-27T12:43:00.992 INFO:tasks.rados.rados.0.smithi138.stdout:1173: finishing write tid 5 to smithi138138488-45 2023-06-27T12:43:00.992 INFO:tasks.rados.rados.0.smithi138.stdout:1173: oid 45 updating version 0 to 368 2023-06-27T12:43:00.993 INFO:tasks.rados.rados.0.smithi138.stdout:1173: oid 45 version 368 is already newer than 368 2023-06-27T12:43:00.993 INFO:tasks.rados.rados.0.smithi138.stdout:1173: oid 45 version 368 is already newer than 364 2023-06-27T12:43:00.993 INFO:tasks.rados.rados.0.smithi138.stdout:1173: oid 45 version 368 is already newer than 368 2023-06-27T12:43:00.993 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 368 (ObjNum 568 snap 133 seq_num 568) dirty exists 2023-06-27T12:43:00.993 INFO:tasks.rados.rados.0.smithi138.stdout:1173: left oid 45 (ObjNum 568 snap 133 seq_num 568) 2023-06-27T12:43:01.520 INFO:tasks.rados.rados.0.smithi138.stdout:1173: done (1 left) 2023-06-27T12:43:01.520 INFO:tasks.rados.rados.0.smithi138.stdout:1176: done (0 left) 2023-06-27T12:43:01.521 INFO:tasks.rados.rados.0.smithi138.stdout:1177: write (excl) oid 9 current snap is 134 2023-06-27T12:43:01.521 INFO:tasks.rados.rados.0.smithi138.stdout:1177: seq_num 570 ranges {779990=702052,2055442=653193,3253694=326296} 2023-06-27T12:43:01.528 INFO:tasks.rados.rados.0.smithi138.stdout:1177: writing smithi138138488-9 from 779990 to 1482042 tid 1 2023-06-27T12:43:01.538 INFO:tasks.rados.rados.0.smithi138.stdout:1177: writing smithi138138488-9 from 2055442 to 2708635 tid 2 2023-06-27T12:43:01.544 INFO:tasks.rados.rados.0.smithi138.stdout:1177: writing smithi138138488-9 from 3253694 to 3579990 tid 3 2023-06-27T12:43:01.544 INFO:tasks.rados.rados.0.smithi138.stdout:1178: delete oid 39 current snap is 134 2023-06-27T12:43:01.559 INFO:tasks.rados.rados.0.smithi138.stdout:1178: done (1 left) 2023-06-27T12:43:01.559 INFO:tasks.rados.rados.0.smithi138.stdout:1179: read oid 19 snap -1 2023-06-27T12:43:01.559 INFO:tasks.rados.rados.0.smithi138.stdout:1179: expect deleted 2023-06-27T12:43:01.559 INFO:tasks.rados.rados.0.smithi138.stdout:1180: snap_remove snap 128 2023-06-27T12:43:02.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:02.409 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:02.404+0000 7fd28f04c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:02.412 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:02.408+0000 7fd28f04c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:02.457 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:02.460 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:02.460 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:02.524 INFO:tasks.rados.rados.0.smithi138.stdout:1180: done (2 left) 2023-06-27T12:43:02.524 INFO:tasks.rados.rados.0.smithi138.stdout:1181: snap_create 2023-06-27T12:43:02.525 INFO:tasks.rados.rados.0.smithi138.stdout:1177: finishing write tid 1 to smithi138138488-9 2023-06-27T12:43:02.525 INFO:tasks.rados.rados.0.smithi138.stdout:1177: finishing write tid 2 to smithi138138488-9 2023-06-27T12:43:02.525 INFO:tasks.rados.rados.0.smithi138.stdout:1177: finishing write tid 3 to smithi138138488-9 2023-06-27T12:43:02.525 INFO:tasks.rados.rados.0.smithi138.stdout:1177: finishing write tid 4 to smithi138138488-9 2023-06-27T12:43:02.525 INFO:tasks.rados.rados.0.smithi138.stdout:1177: finishing write tid 5 to smithi138138488-9 2023-06-27T12:43:02.525 INFO:tasks.rados.rados.0.smithi138.stdout:1177: oid 9 updating version 0 to 347 2023-06-27T12:43:02.525 INFO:tasks.rados.rados.0.smithi138.stdout:1177: oid 9 version 347 is already newer than 347 2023-06-27T12:43:02.526 INFO:tasks.rados.rados.0.smithi138.stdout:1177: oid 9 version 347 is already newer than 347 2023-06-27T12:43:02.526 INFO:tasks.rados.rados.0.smithi138.stdout:1177: oid 9 version 347 is already newer than 343 2023-06-27T12:43:02.526 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 347 (ObjNum 570 snap 134 seq_num 570) dirty exists 2023-06-27T12:43:02.526 INFO:tasks.rados.rados.0.smithi138.stdout:1177: left oid 9 (ObjNum 570 snap 134 seq_num 570) 2023-06-27T12:43:03.526 INFO:tasks.rados.rados.0.smithi138.stdout:1177: done (2 left) 2023-06-27T12:43:03.527 INFO:tasks.rados.rados.0.smithi138.stdout:1179: done (1 left) 2023-06-27T12:43:03.527 INFO:tasks.rados.rados.0.smithi138.stdout:1181: done (0 left) 2023-06-27T12:43:03.527 INFO:tasks.rados.rados.0.smithi138.stdout:1182: snap_create 2023-06-27T12:43:03.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~149s 2023-06-27T12:43:04.461 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:04.533 INFO:tasks.rados.rados.0.smithi138.stdout:1182: done (0 left) 2023-06-27T12:43:04.533 INFO:tasks.rados.rados.0.smithi138.stdout:1183: write oid 23 current snap is 136 2023-06-27T12:43:04.534 INFO:tasks.rados.rados.0.smithi138.stdout:1183: seq_num 572 ranges {413604=568903,1488759=656862,2624893=591709} 2023-06-27T12:43:04.539 INFO:tasks.rados.rados.0.smithi138.stdout:1183: writing smithi138138488-23 from 413604 to 982507 tid 1 2023-06-27T12:43:04.545 INFO:tasks.rados.rados.0.smithi138.stdout:1183: writing smithi138138488-23 from 1488759 to 2145621 tid 2 2023-06-27T12:43:04.553 INFO:tasks.rados.rados.0.smithi138.stdout:1183: writing smithi138138488-23 from 2624893 to 3216602 tid 3 2023-06-27T12:43:04.553 INFO:tasks.rados.rados.0.smithi138.stdout:1184: snap_create 2023-06-27T12:43:04.607 INFO:tasks.rados.rados.0.smithi138.stdout:1183: finishing write tid 1 to smithi138138488-23 2023-06-27T12:43:04.613 INFO:tasks.rados.rados.0.smithi138.stdout:1183: finishing write tid 2 to smithi138138488-23 2023-06-27T12:43:04.629 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:04.624+0000 7f9a9702e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:04.632 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:04.628+0000 7f9a9702e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:04.656 INFO:tasks.rados.rados.0.smithi138.stdout:1183: finishing write tid 3 to smithi138138488-23 2023-06-27T12:43:04.656 INFO:tasks.rados.rados.0.smithi138.stdout:1183: finishing write tid 4 to smithi138138488-23 2023-06-27T12:43:04.658 INFO:tasks.rados.rados.0.smithi138.stdout:1183: finishing write tid 5 to smithi138138488-23 2023-06-27T12:43:04.658 INFO:tasks.rados.rados.0.smithi138.stdout:1183: oid 23 updating version 0 to 167 2023-06-27T12:43:04.658 INFO:tasks.rados.rados.0.smithi138.stdout:1183: oid 23 version 167 is already newer than 167 2023-06-27T12:43:04.658 INFO:tasks.rados.rados.0.smithi138.stdout:1183: oid 23 version 167 is already newer than 167 2023-06-27T12:43:04.658 INFO:tasks.rados.rados.0.smithi138.stdout:1183: oid 23 version 167 is already newer than 167 2023-06-27T12:43:04.658 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 167 (ObjNum 572 snap 136 seq_num 572) dirty exists 2023-06-27T12:43:04.659 INFO:tasks.rados.rados.0.smithi138.stdout:1183: left oid 23 (ObjNum 572 snap 136 seq_num 572) 2023-06-27T12:43:04.677 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:04.680 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:04.680 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:04.985 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:04.986 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:04.986 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:05.106+0000 7faf00ae8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:05.116 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:05.113+0000 7faf00ae8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:05.311 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:05.313 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:05.324 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2023-06-27T12:43:04.528346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":104609903,"num_objects":287,"num_object_clones":241,"num_object_copies":861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":21,"num_read":1738,"num_read_kb":1252979,"num_write":1549,"num_write_kb":449792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2866,"ondisk_log_size":2866,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":884128,"kb_used_data":748160,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376603232,"statfs":{"total":386547056640,"available":385641709568,"internally_reserved":0,"allocated":766115840,"data_stored":864198340,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5428826,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":16,"num_read_kb":13554,"num_write":20,"num_write_kb":6187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017603"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":694,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.818133+0000","last_change":"2023-06-27T12:43:02.534242+0000","last_active":"2023-06-27T12:43:03.818133+0000","last_peered":"2023-06-27T12:43:03.818133+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:03.818133+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"283'174","reported_seq":911,"reported_epoch":289,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.826748+0000","last_change":"2023-06-27T12:43:03.827278+0000","last_active":"2023-06-27T12:43:03.826748+0000","last_peered":"2023-06-27T12:43:03.826748+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:03.826748+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10756711,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":127,"num_read_kb":152683,"num_write":124,"num_write_kb":36349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"282'238","reported_seq":911,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.554970+0000","last_change":"2023-06-27T12:43:03.555415+0000","last_active":"2023-06-27T12:43:03.554970+0000","last_peered":"2023-06-27T12:43:03.554970+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:03.554970+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278025,"num_objects":27,"num_object_clones":22,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":113115,"num_write":165,"num_write_kb":47622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":27}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"268'111","reported_seq":837,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.811045+0000","last_change":"2023-06-27T12:43:02.536882+0000","last_active":"2023-06-27T12:43:03.811045+0000","last_peered":"2023-06-27T12:43:03.811045+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:03.811045+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9529833,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":121,"num_read_kb":80249,"num_write":88,"num_write_kb":24907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"286'82","reported_seq":406,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.815586+0000","last_change":"2023-06-27T12:43:02.535251+0000","last_active":"2023-06-27T12:43:03.815586+0000","last_peered":"2023-06-27T12:43:03.815586+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:03.815586+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6437279,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":40838,"num_write":44,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"284'363","reported_seq":939,"reported_epoch":290,"state":"active+clean","last_fresh":"2023-06-27T12:43:03.811501+0000","last_change":"2023-06-27T12:43:02.580969+0000","last_active":"2023-06-27T12:43:03.811501+0000","last_peered":"2023-06-27T12:43:03.811501+0000","last_clean":"2023-06-27T12:43:03.811501+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:03.811501+0000","last_undegraded":"2023-06-27T12:43:03.811501+0000","last_fullsized":"2023-06-27T12:43:03.811501+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13748489,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":283,"num_read_kb":175221,"num_write":220,"num_write_kb":63975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"283'159","reported_seq":742,"reported_epoch":290,"state":"active+clean","last_fresh":"2023-06-27T12:43:03.817771+0000","last_change":"2023-06-27T12:43:02.536619+0000","last_active":"2023-06-27T12:43:03.817771+0000","last_peered":"2023-06-27T12:43:03.817771+0000","last_clean":"2023-06-27T12:43:03.817771+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:03.817771+0000","last_undegraded":"2023-06-27T12:43:03.817771+0000","last_fullsized":"2023-06-27T12:43:03.817771+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395113,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":111,"num_read_kb":78356,"num_write":90,"num_write_kb":26241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":733,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.543580+0000","last_change":"2023-06-27T12:43:03.551915+0000","last_active":"2023-06-27T12:43:03.543580+0000","last_peered":"2023-06-27T12:43:03.543580+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:03.543580+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":482,"reported_epoch":289,"state":"active+undersized","last_fresh":"2023-06-27T12:43:03.816226+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:03.816226+0000","last_peered":"2023-06-27T12:43:03.816226+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:03.816226+0000","last_undegraded":"2023-06-27T12:43:03.816226+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":647,"reported_epoch":290,"state":"active+undersized","last_fresh":"2023-06-27T12:43:03.547380+0000","last_change":"2023-06-27T12:43:03.553113+0000","last_active":"2023-06-27T12:43:03.547380+0000","last_peered":"2023-06-27T12:43:03.547380+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:03.547380+0000","last_undegraded":"2023-06-27T12:43:03.547380+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":461,"reported_epoch":290,"state":"active+clean","last_fresh":"2023-06-27T12:43:03.819626+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:03.819626+0000","last_peered":"2023-06-27T12:43:03.819626+0000","last_clean":"2023-06-27T12:43:03.819626+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:03.819626+0000","last_undegraded":"2023-06-27T12:43:03.819626+0000","last_fullsized":"2023-06-27T12:43:03.819626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"261'150","reported_seq":836,"reported_epoch":289,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.810999+0000","last_change":"2023-06-27T12:43:03.824007+0000","last_active":"2023-06-27T12:43:03.810999+0000","last_peered":"2023-06-27T12:43:03.810999+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:03.810999+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563315,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":66415,"num_write":101,"num_write_kb":27991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":450,"reported_epoch":290,"state":"active+undersized","last_fresh":"2023-06-27T12:43:03.540895+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:03.540895+0000","last_peered":"2023-06-27T12:43:03.540895+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:03.540895+0000","last_undegraded":"2023-06-27T12:43:03.540895+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"286'368","reported_seq":487,"reported_epoch":289,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.827840+0000","last_change":"2023-06-27T12:43:03.828302+0000","last_active":"2023-06-27T12:43:03.827840+0000","last_peered":"2023-06-27T12:43:03.827840+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:03.827840+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4890969,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":47,"num_read_kb":29628,"num_write":83,"num_write_kb":25364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":16}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":447,"reported_epoch":290,"state":"active+clean","last_fresh":"2023-06-27T12:43:03.547615+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:03.547615+0000","last_peered":"2023-06-27T12:43:03.547615+0000","last_clean":"2023-06-27T12:43:03.547615+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:03.547615+0000","last_undegraded":"2023-06-27T12:43:03.547615+0000","last_fullsized":"2023-06-27T12:43:03.547615+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":213,"reported_epoch":290,"state":"active+undersized","last_fresh":"2023-06-27T12:43:03.818903+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:03.818903+0000","last_peered":"2023-06-27T12:43:03.818903+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:03.818903+0000","last_undegraded":"2023-06-27T12:43:03.818903+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"288'347","reported_seq":1041,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.553491+0000","last_change":"2023-06-27T12:43:03.553902+0000","last_active":"2023-06-27T12:43:03.553491+0000","last_peered":"2023-06-27T12:43:03.553491+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:03.553491+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9076077,"num_objects":44,"num_object_clones":38,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":284,"num_read_kb":173900,"num_write":252,"num_write_kb":73394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":44}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":461,"reported_epoch":290,"state":"active+clean","last_fresh":"2023-06-27T12:43:03.818909+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:03.818909+0000","last_peered":"2023-06-27T12:43:03.818909+0000","last_clean":"2023-06-27T12:43:03.818909+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:03.818909+0000","last_undegraded":"2023-06-27T12:43:03.818909+0000","last_fullsized":"2023-06-27T12:43:03.818909+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"279'189","reported_seq":415,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.537209+0000","last_change":"2023-06-27T12:43:03.550608+0000","last_active":"2023-06-27T12:43:03.537209+0000","last_peered":"2023-06-27T12:43:03.537209+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:03.537209+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3227043,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":41527,"num_write":54,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":225,"reported_epoch":289,"state":"active+undersized","last_fresh":"2023-06-27T12:43:03.809564+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:03.809564+0000","last_peered":"2023-06-27T12:43:03.809564+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:03.809564+0000","last_undegraded":"2023-06-27T12:43:03.809564+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"286'173","reported_seq":871,"reported_epoch":289,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.824720+0000","last_change":"2023-06-27T12:43:03.825205+0000","last_active":"2023-06-27T12:43:03.824720+0000","last_peered":"2023-06-27T12:43:03.824720+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:03.824720+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6134753,"num_objects":23,"num_object_clones":19,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":93,"num_read_kb":105200,"num_write":114,"num_write_kb":33633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":403,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.543364+0000","last_change":"2023-06-27T12:43:03.550805+0000","last_active":"2023-06-27T12:43:03.543364+0000","last_peered":"2023-06-27T12:43:03.543364+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:03.543364+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":48530,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":482,"reported_epoch":289,"state":"active+undersized","last_fresh":"2023-06-27T12:43:03.820798+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:03.820798+0000","last_peered":"2023-06-27T12:43:03.820798+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:03.820798+0000","last_undegraded":"2023-06-27T12:43:03.820798+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":447,"reported_epoch":290,"state":"active+clean","last_fresh":"2023-06-27T12:43:03.539954+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:03.539954+0000","last_peered":"2023-06-27T12:43:03.539954+0000","last_clean":"2023-06-27T12:43:03.539954+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:03.539954+0000","last_undegraded":"2023-06-27T12:43:03.539954+0000","last_fullsized":"2023-06-27T12:43:03.539954+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"283'207","reported_seq":878,"reported_epoch":289,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:03.822813+0000","last_change":"2023-06-27T12:43:03.823335+0000","last_active":"2023-06-27T12:43:03.822813+0000","last_peered":"2023-06-27T12:43:03.822813+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:03.822813+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6571735,"num_objects":23,"num_object_clones":20,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":102,"num_read_kb":100184,"num_write":111,"num_write_kb":32264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":104609903,"num_objects":287,"num_object_clones":241,"num_object_copies":861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":21,"num_read":1738,"num_read_kb":1252979,"num_write":1549,"num_write_kb":449792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2769,"ondisk_log_size":2769,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673270,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":221744,"kb_used_data":186728,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94150096,"statfs":{"total":96636764160,"available":96409698304,"internally_reserved":0,"allocated":191209472,"data_stored":214594931,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":229644,"kb_used_data":194628,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94142196,"statfs":{"total":96636764160,"available":96401608704,"internally_reserved":0,"allocated":199299072,"data_stored":222660672,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673270,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":251460,"kb_used_data":216444,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94120380,"statfs":{"total":96636764160,"available":96379269120,"internally_reserved":0,"allocated":221638656,"data_stored":249903185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:05.326 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.d 3.b 3.8 3.2 3.1 3.4 3.7 2023-06-27T12:43:05.440 INFO:tasks.rados.rados.0.smithi138.stdout:1183: done (1 left) 2023-06-27T12:43:05.440 INFO:tasks.rados.rados.0.smithi138.stdout:1184: done (0 left) 2023-06-27T12:43:05.440 INFO:tasks.rados.rados.0.smithi138.stdout:1185: snap_remove snap 135 2023-06-27T12:43:05.463 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:05.447+0000 7fe3f6361700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:05.465 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:05.462+0000 7fe3f6361700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:05.664 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.f] on osd.0 to force-backfill; instructing pg(s) [3.1,3.2,3.7,3.8,3.d] on osd.1 to force-backfill; instructing pg(s) [3.4] on osd.2 to force-backfill; 2023-06-27T12:43:05.664 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:06.443 INFO:tasks.rados.rados.0.smithi138.stdout:1185: done (0 left) 2023-06-27T12:43:06.444 INFO:tasks.rados.rados.0.smithi138.stdout:1186: read oid 15 snap 134 2023-06-27T12:43:06.444 INFO:tasks.rados.rados.0.smithi138.stdout:1186: expect (ObjNum 567 snap 133 seq_num 567) 2023-06-27T12:43:06.444 INFO:tasks.rados.rados.0.smithi138.stdout:1187: read oid 4 snap 134 2023-06-27T12:43:06.444 INFO:tasks.rados.rados.0.smithi138.stdout:1187: expect (ObjNum 421 snap 99 seq_num 421) 2023-06-27T12:43:06.444 INFO:tasks.rados.rados.0.smithi138.stdout:1188: write oid 39 current snap is 137 2023-06-27T12:43:06.444 INFO:tasks.rados.rados.0.smithi138.stdout:1188: seq_num 573 ranges {430411=644152,1830888=400520,2935963=402981} 2023-06-27T12:43:06.449 INFO:tasks.rados.rados.0.smithi138.stdout:1188: writing smithi138138488-39 from 430411 to 1074563 tid 1 2023-06-27T12:43:06.457 INFO:tasks.rados.rados.0.smithi138.stdout:1188: writing smithi138138488-39 from 1830888 to 2231408 tid 2 2023-06-27T12:43:06.464 INFO:tasks.rados.rados.0.smithi138.stdout:1188: writing smithi138138488-39 from 2935963 to 3338944 tid 3 2023-06-27T12:43:06.465 INFO:tasks.rados.rados.0.smithi138.stdout:1189: snap_create 2023-06-27T12:43:06.531 INFO:tasks.rados.rados.0.smithi138.stdout:1186: expect (ObjNum 567 snap 133 seq_num 567) 2023-06-27T12:43:06.682 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:06.710 INFO:tasks.rados.rados.0.smithi138.stdout:1187: expect (ObjNum 421 snap 99 seq_num 421) 2023-06-27T12:43:06.808 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:06.803+0000 7f777f697700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:06.811 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:06.807+0000 7f777f697700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:06.859 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:06.860 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:06.916 INFO:tasks.rados.rados.0.smithi138.stdout:1188: finishing write tid 1 to smithi138138488-39 2023-06-27T12:43:06.917 INFO:tasks.rados.rados.0.smithi138.stdout:1188: finishing write tid 2 to smithi138138488-39 2023-06-27T12:43:06.917 INFO:tasks.rados.rados.0.smithi138.stdout:1188: finishing write tid 3 to smithi138138488-39 2023-06-27T12:43:06.917 INFO:tasks.rados.rados.0.smithi138.stdout:1188: finishing write tid 4 to smithi138138488-39 2023-06-27T12:43:06.917 INFO:tasks.rados.rados.0.smithi138.stdout:1188: finishing write tid 5 to smithi138138488-39 2023-06-27T12:43:06.917 INFO:tasks.rados.rados.0.smithi138.stdout:1188: oid 39 updating version 0 to 196 2023-06-27T12:43:06.917 INFO:tasks.rados.rados.0.smithi138.stdout:1188: oid 39 version 196 is already newer than 196 2023-06-27T12:43:06.918 INFO:tasks.rados.rados.0.smithi138.stdout:1188: oid 39 version 196 is already newer than 196 2023-06-27T12:43:06.918 INFO:tasks.rados.rados.0.smithi138.stdout:1188: oid 39 version 196 is already newer than 196 2023-06-27T12:43:06.918 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 39 v 196 (ObjNum 573 snap 137 seq_num 573) dirty exists 2023-06-27T12:43:06.918 INFO:tasks.rados.rados.0.smithi138.stdout:1188: left oid 39 (ObjNum 573 snap 137 seq_num 573) 2023-06-27T12:43:07.442 INFO:tasks.rados.rados.0.smithi138.stdout:1186: done (3 left) 2023-06-27T12:43:07.442 INFO:tasks.rados.rados.0.smithi138.stdout:1187: done (2 left) 2023-06-27T12:43:07.443 INFO:tasks.rados.rados.0.smithi138.stdout:1188: done (1 left) 2023-06-27T12:43:07.443 INFO:tasks.rados.rados.0.smithi138.stdout:1189: done (0 left) 2023-06-27T12:43:07.443 INFO:tasks.rados.rados.0.smithi138.stdout:1190: snap_remove snap 136 2023-06-27T12:43:08.443 INFO:tasks.rados.rados.0.smithi138.stdout:1190: done (0 left) 2023-06-27T12:43:08.444 INFO:tasks.rados.rados.0.smithi138.stdout:1191: write (excl) oid 32 current snap is 138 2023-06-27T12:43:08.444 INFO:tasks.rados.rados.0.smithi138.stdout:1191: seq_num 574 ranges {447218=719401,1889370=660531,2963386=614253} 2023-06-27T12:43:08.452 INFO:tasks.rados.rados.0.smithi138.stdout:1191: writing smithi138138488-32 from 447218 to 1166619 tid 1 2023-06-27T12:43:08.460 INFO:tasks.rados.rados.0.smithi138.stdout:1191: writing smithi138138488-32 from 1889370 to 2549901 tid 2 2023-06-27T12:43:08.468 INFO:tasks.rados.rados.0.smithi138.stdout:1191: writing smithi138138488-32 from 2963386 to 3577639 tid 3 2023-06-27T12:43:08.468 INFO:tasks.rados.rados.0.smithi138.stdout:1192: snap_remove snap 134 2023-06-27T12:43:08.861 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:08.990 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:08.985+0000 7f1d28f98700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:08.989+0000 7f1d28f98700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:09.037 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:09.039 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:09.040 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:09.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~154s 2023-06-27T12:43:09.446 INFO:tasks.rados.rados.0.smithi138.stdout:1192: done (1 left) 2023-06-27T12:43:09.446 INFO:tasks.rados.rados.0.smithi138.stdout:1193: snap_remove snap 130 2023-06-27T12:43:10.452 INFO:tasks.rados.rados.0.smithi138.stdout:1193: done (1 left) 2023-06-27T12:43:10.453 INFO:tasks.rados.rados.0.smithi138.stdout:1194: delete oid 37 current snap is 138 2023-06-27T12:43:10.453 INFO:tasks.rados.rados.0.smithi138.stdout:1191: finishing write tid 1 to smithi138138488-32 2023-06-27T12:43:10.453 INFO:tasks.rados.rados.0.smithi138.stdout:1191: finishing write tid 2 to smithi138138488-32 2023-06-27T12:43:10.453 INFO:tasks.rados.rados.0.smithi138.stdout:1191: finishing write tid 3 to smithi138138488-32 2023-06-27T12:43:10.454 INFO:tasks.rados.rados.0.smithi138.stdout:1191: finishing write tid 4 to smithi138138488-32 2023-06-27T12:43:10.454 INFO:tasks.rados.rados.0.smithi138.stdout:1191: finishing write tid 5 to smithi138138488-32 2023-06-27T12:43:10.454 INFO:tasks.rados.rados.0.smithi138.stdout:1191: oid 32 updating version 0 to 377 2023-06-27T12:43:10.454 INFO:tasks.rados.rados.0.smithi138.stdout:1191: oid 32 version 377 is already newer than 377 2023-06-27T12:43:10.454 INFO:tasks.rados.rados.0.smithi138.stdout:1191: oid 32 version 377 is already newer than 377 2023-06-27T12:43:10.454 INFO:tasks.rados.rados.0.smithi138.stdout:1191: oid 32 version 377 is already newer than 373 2023-06-27T12:43:10.455 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 377 (ObjNum 574 snap 138 seq_num 574) dirty exists 2023-06-27T12:43:10.455 INFO:tasks.rados.rados.0.smithi138.stdout:1191: left oid 32 (ObjNum 574 snap 138 seq_num 574) 2023-06-27T12:43:10.502 INFO:tasks.rados.rados.0.smithi138.stdout:1191: done (1 left) 2023-06-27T12:43:10.503 INFO:tasks.rados.rados.0.smithi138.stdout:1194: done (0 left) 2023-06-27T12:43:10.503 INFO:tasks.rados.rados.0.smithi138.stdout:1195: write oid 25 current snap is 138 2023-06-27T12:43:10.503 INFO:tasks.rados.rados.0.smithi138.stdout:1195: seq_num 576 ranges {480832=469899,1606334=664200,3018232=662600} 2023-06-27T12:43:10.507 INFO:tasks.rados.rados.0.smithi138.stdout:1195: writing smithi138138488-25 from 480832 to 950731 tid 1 2023-06-27T12:43:10.517 INFO:tasks.rados.rados.0.smithi138.stdout:1195: writing smithi138138488-25 from 1606334 to 2270534 tid 2 2023-06-27T12:43:10.525 INFO:tasks.rados.rados.0.smithi138.stdout:1195: writing smithi138138488-25 from 3018232 to 3680832 tid 3 2023-06-27T12:43:10.525 INFO:tasks.rados.rados.0.smithi138.stdout:1196: read oid 48 snap -1 2023-06-27T12:43:10.526 INFO:tasks.rados.rados.0.smithi138.stdout:1196: expect (ObjNum 544 snap 128 seq_num 544) 2023-06-27T12:43:10.526 INFO:tasks.rados.rados.0.smithi138.stdout:1197: delete oid 7 current snap is 138 2023-06-27T12:43:10.562 INFO:tasks.rados.rados.0.smithi138.stdout:1195: finishing write tid 1 to smithi138138488-25 2023-06-27T12:43:10.570 INFO:tasks.rados.rados.0.smithi138.stdout:1196: expect (ObjNum 544 snap 128 seq_num 544) 2023-06-27T12:43:10.676 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:10.677 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:10.677 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:10.792 INFO:tasks.rados.rados.0.smithi138.stdout:1195: finishing write tid 2 to smithi138138488-25 2023-06-27T12:43:10.792 INFO:tasks.rados.rados.0.smithi138.stdout:1195: finishing write tid 3 to smithi138138488-25 2023-06-27T12:43:10.792 INFO:tasks.rados.rados.0.smithi138.stdout:1195: finishing write tid 4 to smithi138138488-25 2023-06-27T12:43:10.793 INFO:tasks.rados.rados.0.smithi138.stdout:1196: done (2 left) 2023-06-27T12:43:10.793 INFO:tasks.rados.rados.0.smithi138.stdout:1197: done (1 left) 2023-06-27T12:43:10.793 INFO:tasks.rados.rados.0.smithi138.stdout:1198: write (excl) oid 22 current snap is 138 2023-06-27T12:43:10.793 INFO:tasks.rados.rados.0.smithi138.stdout:1195: finishing write tid 5 to smithi138138488-25 2023-06-27T12:43:10.793 INFO:tasks.rados.rados.0.smithi138.stdout:1195: oid 25 updating version 0 to 109 2023-06-27T12:43:10.793 INFO:tasks.rados.rados.0.smithi138.stdout:1195: oid 25 version 109 is already newer than 109 2023-06-27T12:43:10.793 INFO:tasks.rados.rados.0.smithi138.stdout:1195: oid 25 version 109 is already newer than 109 2023-06-27T12:43:10.793 INFO:tasks.rados.rados.0.smithi138.stdout:1195: oid 25 version 109 is already newer than 109 2023-06-27T12:43:10.794 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 109 (ObjNum 576 snap 138 seq_num 576) dirty exists 2023-06-27T12:43:10.794 INFO:tasks.rados.rados.0.smithi138.stdout:1195: left oid 25 (ObjNum 576 snap 138 seq_num 576) 2023-06-27T12:43:10.794 INFO:tasks.rados.rados.0.smithi138.stdout:1198: seq_num 578 ranges {514446=736750,1723298=667869,3073078=641368} 2023-06-27T12:43:10.799 INFO:tasks.rados.rados.0.smithi138.stdout:1198: writing smithi138138488-22 from 514446 to 1251196 tid 1 2023-06-27T12:43:10.805 INFO:tasks.rados.rados.0.smithi138.stdout:1198: writing smithi138138488-22 from 1723298 to 2391167 tid 2 2023-06-27T12:43:10.805 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:10.798+0000 7f3cc6059700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:10.809 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:10.805+0000 7f3cc6059700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:10.814 INFO:tasks.rados.rados.0.smithi138.stdout:1198: writing smithi138138488-22 from 3073078 to 3714446 tid 3 2023-06-27T12:43:10.814 INFO:tasks.rados.rados.0.smithi138.stdout:1195: done (1 left) 2023-06-27T12:43:10.814 INFO:tasks.rados.rados.0.smithi138.stdout:1199: snap_create 2023-06-27T12:43:10.852 INFO:tasks.rados.rados.0.smithi138.stdout:1198: finishing write tid 1 to smithi138138488-22 2023-06-27T12:43:10.903 INFO:tasks.rados.rados.0.smithi138.stdout:1198: finishing write tid 2 to smithi138138488-22 2023-06-27T12:43:10.903 INFO:tasks.rados.rados.0.smithi138.stdout:1198: finishing write tid 3 to smithi138138488-22 2023-06-27T12:43:10.904 INFO:tasks.rados.rados.0.smithi138.stdout:1198: finishing write tid 4 to smithi138138488-22 2023-06-27T12:43:10.905 INFO:tasks.rados.rados.0.smithi138.stdout:1198: finishing write tid 5 to smithi138138488-22 2023-06-27T12:43:10.906 INFO:tasks.rados.rados.0.smithi138.stdout:1198: oid 22 updating version 0 to 210 2023-06-27T12:43:10.906 INFO:tasks.rados.rados.0.smithi138.stdout:1198: oid 22 updating version 210 to 214 2023-06-27T12:43:10.906 INFO:tasks.rados.rados.0.smithi138.stdout:1198: oid 22 version 214 is already newer than 214 2023-06-27T12:43:10.906 INFO:tasks.rados.rados.0.smithi138.stdout:1198: oid 22 version 214 is already newer than 214 2023-06-27T12:43:10.906 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 214 (ObjNum 578 snap 138 seq_num 578) dirty exists 2023-06-27T12:43:10.906 INFO:tasks.rados.rados.0.smithi138.stdout:1198: left oid 22 (ObjNum 578 snap 138 seq_num 578) 2023-06-27T12:43:11.027 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:11.029 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:11.041 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2023-06-27T12:43:10.446725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110458871,"num_objects":289,"num_object_clones":243,"num_object_copies":867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1756,"num_read_kb":1268163,"num_write":1564,"num_write_kb":454933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2899,"ondisk_log_size":2899,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":894600,"kb_used_data":758632,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376592760,"statfs":{"total":386547056640,"available":385630986240,"internally_reserved":0,"allocated":776839168,"data_stored":877512704,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5848968,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":18,"num_read_kb":15184,"num_write":15,"num_write_kb":5141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.008541"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":704,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.526409+0000","last_change":"2023-06-27T12:43:08.529402+0000","last_active":"2023-06-27T12:43:08.526409+0000","last_peered":"2023-06-27T12:43:08.526409+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:08.526409+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"283'174","reported_seq":919,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.506362+0000","last_change":"2023-06-27T12:43:08.511926+0000","last_active":"2023-06-27T12:43:08.506362+0000","last_peered":"2023-06-27T12:43:08.506362+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:08.506362+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10756711,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":127,"num_read_kb":152683,"num_write":124,"num_write_kb":36349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"282'238","reported_seq":919,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.466899+0000","last_change":"2023-06-27T12:43:08.474992+0000","last_active":"2023-06-27T12:43:08.466899+0000","last_peered":"2023-06-27T12:43:08.466899+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:08.466899+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278025,"num_objects":27,"num_object_clones":22,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":113115,"num_write":165,"num_write_kb":47622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":27}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"268'111","reported_seq":847,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.531346+0000","last_change":"2023-06-27T12:43:08.544031+0000","last_active":"2023-06-27T12:43:08.531346+0000","last_peered":"2023-06-27T12:43:08.531346+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:08.531346+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9529833,"num_objects":19,"num_object_clones":15,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":121,"num_read_kb":80249,"num_write":88,"num_write_kb":24907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"286'82","reported_seq":419,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.519542+0000","last_change":"2023-06-27T12:43:08.529877+0000","last_active":"2023-06-27T12:43:08.519542+0000","last_peered":"2023-06-27T12:43:08.519542+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:08.519542+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6437279,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":50181,"num_write":44,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"284'363","reported_seq":949,"reported_epoch":295,"state":"active+clean","last_fresh":"2023-06-27T12:43:08.522627+0000","last_change":"2023-06-27T12:43:08.530536+0000","last_active":"2023-06-27T12:43:08.522627+0000","last_peered":"2023-06-27T12:43:08.522627+0000","last_clean":"2023-06-27T12:43:08.522627+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:08.522627+0000","last_undegraded":"2023-06-27T12:43:08.522627+0000","last_fullsized":"2023-06-27T12:43:08.522627+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13748489,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":283,"num_read_kb":175221,"num_write":220,"num_write_kb":63975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"283'159","reported_seq":752,"reported_epoch":295,"state":"active+clean","last_fresh":"2023-06-27T12:43:08.525029+0000","last_change":"2023-06-27T12:43:08.542321+0000","last_active":"2023-06-27T12:43:08.525029+0000","last_peered":"2023-06-27T12:43:08.525029+0000","last_clean":"2023-06-27T12:43:08.525029+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:08.525029+0000","last_undegraded":"2023-06-27T12:43:08.525029+0000","last_fullsized":"2023-06-27T12:43:08.525029+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395113,"num_objects":19,"num_object_clones":16,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":111,"num_read_kb":78356,"num_write":90,"num_write_kb":26241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":741,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.463603+0000","last_change":"2023-06-27T12:43:08.470884+0000","last_active":"2023-06-27T12:43:08.463603+0000","last_peered":"2023-06-27T12:43:08.463603+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:08.463603+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":488,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-06-27T12:43:08.479420+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:08.479420+0000","last_peered":"2023-06-27T12:43:08.479420+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:08.479420+0000","last_undegraded":"2023-06-27T12:43:08.479420+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":655,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-06-27T12:43:08.468235+0000","last_change":"2023-06-27T12:43:08.474597+0000","last_active":"2023-06-27T12:43:08.468235+0000","last_peered":"2023-06-27T12:43:08.468235+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:08.468235+0000","last_undegraded":"2023-06-27T12:43:08.468235+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":469,"reported_epoch":295,"state":"active+clean","last_fresh":"2023-06-27T12:43:08.520883+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:08.520883+0000","last_peered":"2023-06-27T12:43:08.520883+0000","last_clean":"2023-06-27T12:43:08.520883+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:08.520883+0000","last_undegraded":"2023-06-27T12:43:08.520883+0000","last_fullsized":"2023-06-27T12:43:08.520883+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"291'167","reported_seq":847,"reported_epoch":295,"state":"active+undersized+degraded+snaptrim","last_fresh":"2023-06-27T12:43:08.480040+0000","last_change":"2023-06-27T12:43:08.480040+0000","last_active":"2023-06-27T12:43:08.480040+0000","last_peered":"2023-06-27T12:43:08.480040+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:08.480040+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420472,"num_objects":19,"num_object_clones":14,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":70,"num_read_kb":66416,"num_write":106,"num_write_kb":29767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":456,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-06-27T12:43:08.469459+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:08.469459+0000","last_peered":"2023-06-27T12:43:08.469459+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:08.469459+0000","last_undegraded":"2023-06-27T12:43:08.469459+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"295'377","reported_seq":500,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.556063+0000","last_change":"2023-06-27T12:43:08.513567+0000","last_active":"2023-06-27T12:43:08.556063+0000","last_peered":"2023-06-27T12:43:08.556063+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:08.556063+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6252369,"num_objects":17,"num_object_clones":16,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":49,"num_read_kb":29629,"num_write":88,"num_write_kb":27313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":17}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":453,"reported_epoch":295,"state":"active+clean","last_fresh":"2023-06-27T12:43:08.469623+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:08.469623+0000","last_peered":"2023-06-27T12:43:08.469623+0000","last_clean":"2023-06-27T12:43:08.469623+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:08.469623+0000","last_undegraded":"2023-06-27T12:43:08.469623+0000","last_fullsized":"2023-06-27T12:43:08.469623+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":221,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-06-27T12:43:08.519482+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:08.519482+0000","last_peered":"2023-06-27T12:43:08.519482+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:08.519482+0000","last_undegraded":"2023-06-27T12:43:08.519482+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"288'347","reported_seq":1049,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.464727+0000","last_change":"2023-06-27T12:43:08.473201+0000","last_active":"2023-06-27T12:43:08.464727+0000","last_peered":"2023-06-27T12:43:08.464727+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:08.464727+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9076077,"num_objects":44,"num_object_clones":38,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":284,"num_read_kb":173900,"num_write":252,"num_write_kb":73394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":44}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":469,"reported_epoch":295,"state":"active+clean","last_fresh":"2023-06-27T12:43:08.518416+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:08.518416+0000","last_peered":"2023-06-27T12:43:08.518416+0000","last_clean":"2023-06-27T12:43:08.518416+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:08.518416+0000","last_undegraded":"2023-06-27T12:43:08.518416+0000","last_fullsized":"2023-06-27T12:43:08.518416+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"293'196","reported_seq":431,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.460902+0000","last_change":"2023-06-27T12:43:08.473822+0000","last_active":"2023-06-27T12:43:08.460902+0000","last_peered":"2023-06-27T12:43:08.460902+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:08.460902+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857454,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":85,"num_read_kb":47366,"num_write":59,"num_write_kb":16910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":231,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-06-27T12:43:08.498844+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:08.498844+0000","last_peered":"2023-06-27T12:43:08.498844+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:08.498844+0000","last_undegraded":"2023-06-27T12:43:08.498844+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"286'173","reported_seq":879,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.493342+0000","last_change":"2023-06-27T12:43:08.495656+0000","last_active":"2023-06-27T12:43:08.493342+0000","last_peered":"2023-06-27T12:43:08.493342+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:08.493342+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6134753,"num_objects":23,"num_object_clones":19,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":93,"num_read_kb":105200,"num_write":114,"num_write_kb":33633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"250'104","reported_seq":411,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.466592+0000","last_change":"2023-06-27T12:43:08.474189+0000","last_active":"2023-06-27T12:43:08.466592+0000","last_peered":"2023-06-27T12:43:08.466592+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:08.466592+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4554895,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":48530,"num_write":34,"num_write_kb":9369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":488,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-06-27T12:43:08.478408+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:08.478408+0000","last_peered":"2023-06-27T12:43:08.478408+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:08.478408+0000","last_undegraded":"2023-06-27T12:43:08.478408+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":453,"reported_epoch":295,"state":"active+clean","last_fresh":"2023-06-27T12:43:08.467712+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:08.467712+0000","last_peered":"2023-06-27T12:43:08.467712+0000","last_clean":"2023-06-27T12:43:08.467712+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:08.467712+0000","last_undegraded":"2023-06-27T12:43:08.467712+0000","last_fullsized":"2023-06-27T12:43:08.467712+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"283'207","reported_seq":886,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:08.486195+0000","last_change":"2023-06-27T12:43:08.488551+0000","last_active":"2023-06-27T12:43:08.486195+0000","last_peered":"2023-06-27T12:43:08.486195+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:08.486195+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6571735,"num_objects":23,"num_object_clones":20,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":102,"num_read_kb":100184,"num_write":111,"num_write_kb":32264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":110458871,"num_objects":289,"num_object_clones":243,"num_object_copies":867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1756,"num_read_kb":1268163,"num_write":1564,"num_write_kb":454933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2802,"ondisk_log_size":2802,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":225536,"kb_used_data":190520,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94146304,"statfs":{"total":96636764160,"available":96405815296,"internally_reserved":0,"allocated":195092480,"data_stored":219796126,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":232900,"kb_used_data":197884,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94138940,"statfs":{"total":96636764160,"available":96398274560,"internally_reserved":0,"allocated":202633216,"data_stored":226717434,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":254884,"kb_used_data":219868,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94116956,"statfs":{"total":96636764160,"available":96375762944,"internally_reserved":0,"allocated":225144832,"data_stored":253959592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:11.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.c 3.3 3.2 3.1 3.7 3.6 2023-06-27T12:43:11.043 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:11.171 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:11.166+0000 7f3a00481700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:11.174 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:11.170+0000 7f3a00481700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:11.182 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:11.166+0000 7ff22865a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:11.185 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:11.181+0000 7ff22865a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:11.229 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:11.229 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c,3.f] on osd.0 to force-recovery; instructing pg(s) [3.1,3.2,3.7] on osd.1 to force-recovery; instructing pg(s) [3.3,3.6] on osd.2 to force-recovery; 2023-06-27T12:43:11.385 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:11.453 INFO:tasks.rados.rados.0.smithi138.stdout:1198: done (1 left) 2023-06-27T12:43:11.453 INFO:tasks.rados.rados.0.smithi138.stdout:1199: done (0 left) 2023-06-27T12:43:11.453 INFO:tasks.rados.rados.0.smithi138.stdout:1200: read oid 31 snap 133 2023-06-27T12:43:11.453 INFO:tasks.rados.rados.0.smithi138.stdout:1200: expect (ObjNum 563 snap 132 seq_num 563) 2023-06-27T12:43:11.454 INFO:tasks.rados.rados.0.smithi138.stdout:1201: write (excl) oid 45 current snap is 139 2023-06-27T12:43:11.454 INFO:tasks.rados.rados.0.smithi138.stdout:1201: seq_num 579 ranges {531253=411999,1381780=411527,2300501=586966,3376974=354279} 2023-06-27T12:43:11.458 INFO:tasks.rados.rados.0.smithi138.stdout:1201: writing smithi138138488-45 from 531253 to 943252 tid 1 2023-06-27T12:43:11.465 INFO:tasks.rados.rados.0.smithi138.stdout:1201: writing smithi138138488-45 from 1381780 to 1793307 tid 2 2023-06-27T12:43:11.473 INFO:tasks.rados.rados.0.smithi138.stdout:1201: writing smithi138138488-45 from 2300501 to 2887467 tid 3 2023-06-27T12:43:11.479 INFO:tasks.rados.rados.0.smithi138.stdout:1201: writing smithi138138488-45 from 3376974 to 3731253 tid 4 2023-06-27T12:43:11.479 INFO:tasks.rados.rados.0.smithi138.stdout:1202: write (excl) oid 17 current snap is 139 2023-06-27T12:43:11.480 INFO:tasks.rados.rados.0.smithi138.stdout:1202: seq_num 580 ranges {548060=487248,1440262=671538,2844277=798238} 2023-06-27T12:43:11.487 INFO:tasks.rados.rados.0.smithi138.stdout:1202: writing smithi138138488-17 from 548060 to 1035308 tid 1 2023-06-27T12:43:11.495 INFO:tasks.rados.rados.0.smithi138.stdout:1202: writing smithi138138488-17 from 1440262 to 2111800 tid 2 2023-06-27T12:43:11.506 INFO:tasks.rados.rados.0.smithi138.stdout:1202: writing smithi138138488-17 from 2844277 to 3642515 tid 3 2023-06-27T12:43:11.506 INFO:tasks.rados.rados.0.smithi138.stdout:1203: write oid 2 current snap is 139 2023-06-27T12:43:11.506 INFO:tasks.rados.rados.0.smithi138.stdout:1203: seq_num 581 ranges {564867=562497,1898744=415196,2755347=725863} 2023-06-27T12:43:11.514 INFO:tasks.rados.rados.0.smithi138.stdout:1203: writing smithi138138488-2 from 564867 to 1127364 tid 1 2023-06-27T12:43:11.521 INFO:tasks.rados.rados.0.smithi138.stdout:1203: writing smithi138138488-2 from 1898744 to 2313940 tid 2 2023-06-27T12:43:11.528 INFO:tasks.rados.rados.0.smithi138.stdout:1203: writing smithi138138488-2 from 2755347 to 3481210 tid 3 2023-06-27T12:43:11.528 INFO:tasks.rados.rados.0.smithi138.stdout:1204: write (excl) oid 33 current snap is 139 2023-06-27T12:43:11.528 INFO:tasks.rados.rados.0.smithi138.stdout:1204: seq_num 582 ranges {581674=637746,1840873=558854,3066417=537135} 2023-06-27T12:43:11.534 INFO:tasks.rados.rados.0.smithi138.stdout:1204: writing smithi138138488-33 from 581674 to 1219420 tid 1 2023-06-27T12:43:11.541 INFO:tasks.rados.rados.0.smithi138.stdout:1204: writing smithi138138488-33 from 1840873 to 2399727 tid 2 2023-06-27T12:43:11.548 INFO:tasks.rados.rados.0.smithi138.stdout:1204: writing smithi138138488-33 from 3066417 to 3603552 tid 3 2023-06-27T12:43:11.548 INFO:tasks.rados.rados.0.smithi138.stdout:1205: snap_remove snap 137 2023-06-27T12:43:12.467 INFO:tasks.rados.rados.0.smithi138.stdout:1205: done (5 left) 2023-06-27T12:43:12.468 INFO:tasks.rados.rados.0.smithi138.stdout:1206: read oid 42 snap -1 2023-06-27T12:43:12.468 INFO:tasks.rados.rados.0.smithi138.stdout:1206: expect (ObjNum 533 snap 122 seq_num 533) 2023-06-27T12:43:12.468 INFO:tasks.rados.rados.0.smithi138.stdout:1207: read oid 22 snap -1 2023-06-27T12:43:12.468 INFO:tasks.rados.rados.0.smithi138.stdout:1207: expect (ObjNum 578 snap 138 seq_num 578) 2023-06-27T12:43:12.468 INFO:tasks.rados.rados.0.smithi138.stdout:1208: delete oid 44 current snap is 139 2023-06-27T12:43:12.468 INFO:tasks.rados.rados.0.smithi138.stdout:1201: finishing write tid 1 to smithi138138488-45 2023-06-27T12:43:12.469 INFO:tasks.rados.rados.0.smithi138.stdout:1200: expect (ObjNum 563 snap 132 seq_num 563) 2023-06-27T12:43:12.658 INFO:tasks.rados.rados.0.smithi138.stdout:1202: finishing write tid 1 to smithi138138488-17 2023-06-27T12:43:12.658 INFO:tasks.rados.rados.0.smithi138.stdout:1201: finishing write tid 2 to smithi138138488-45 2023-06-27T12:43:12.658 INFO:tasks.rados.rados.0.smithi138.stdout:1201: finishing write tid 3 to smithi138138488-45 2023-06-27T12:43:12.659 INFO:tasks.rados.rados.0.smithi138.stdout:1201: finishing write tid 4 to smithi138138488-45 2023-06-27T12:43:12.659 INFO:tasks.rados.rados.0.smithi138.stdout:1201: finishing write tid 5 to smithi138138488-45 2023-06-27T12:43:12.659 INFO:tasks.rados.rados.0.smithi138.stdout:1201: finishing write tid 6 to smithi138138488-45 2023-06-27T12:43:12.659 INFO:tasks.rados.rados.0.smithi138.stdout:1201: oid 45 updating version 0 to 387 2023-06-27T12:43:12.659 INFO:tasks.rados.rados.0.smithi138.stdout:1201: oid 45 version 387 is already newer than 387 2023-06-27T12:43:12.659 INFO:tasks.rados.rados.0.smithi138.stdout:1201: oid 45 version 387 is already newer than 387 2023-06-27T12:43:12.659 INFO:tasks.rados.rados.0.smithi138.stdout:1201: oid 45 version 387 is already newer than 382 2023-06-27T12:43:12.659 INFO:tasks.rados.rados.0.smithi138.stdout:1201: oid 45 version 387 is already newer than 387 2023-06-27T12:43:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 387 (ObjNum 579 snap 139 seq_num 579) dirty exists 2023-06-27T12:43:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1201: left oid 45 (ObjNum 579 snap 139 seq_num 579) 2023-06-27T12:43:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1202: finishing write tid 2 to smithi138138488-17 2023-06-27T12:43:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1200: done (7 left) 2023-06-27T12:43:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1201: done (6 left) 2023-06-27T12:43:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1208: done (5 left) 2023-06-27T12:43:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1209: read oid 36 snap -1 2023-06-27T12:43:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1209: expect (ObjNum 548 snap 130 seq_num 548) 2023-06-27T12:43:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1210: snap_create 2023-06-27T12:43:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1202: finishing write tid 3 to smithi138138488-17 2023-06-27T12:43:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1202: finishing write tid 4 to smithi138138488-17 2023-06-27T12:43:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1203: finishing write tid 1 to smithi138138488-2 2023-06-27T12:43:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1202: finishing write tid 5 to smithi138138488-17 2023-06-27T12:43:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1202: oid 17 updating version 0 to 177 2023-06-27T12:43:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1202: oid 17 updating version 177 to 180 2023-06-27T12:43:12.662 INFO:tasks.rados.rados.0.smithi138.stdout:1202: oid 17 version 180 is already newer than 180 2023-06-27T12:43:12.662 INFO:tasks.rados.rados.0.smithi138.stdout:1202: oid 17 version 180 is already newer than 180 2023-06-27T12:43:12.662 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 180 (ObjNum 580 snap 139 seq_num 580) dirty exists 2023-06-27T12:43:12.662 INFO:tasks.rados.rados.0.smithi138.stdout:1202: left oid 17 (ObjNum 580 snap 139 seq_num 580) 2023-06-27T12:43:12.662 INFO:tasks.rados.rados.0.smithi138.stdout:1203: finishing write tid 2 to smithi138138488-2 2023-06-27T12:43:12.662 INFO:tasks.rados.rados.0.smithi138.stdout:1204: finishing write tid 1 to smithi138138488-33 2023-06-27T12:43:12.662 INFO:tasks.rados.rados.0.smithi138.stdout:1203: finishing write tid 3 to smithi138138488-2 2023-06-27T12:43:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1203: finishing write tid 4 to smithi138138488-2 2023-06-27T12:43:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1203: finishing write tid 5 to smithi138138488-2 2023-06-27T12:43:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1203: oid 2 updating version 0 to 391 2023-06-27T12:43:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1203: oid 2 version 391 is already newer than 391 2023-06-27T12:43:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1203: oid 2 version 391 is already newer than 391 2023-06-27T12:43:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1203: oid 2 version 391 is already newer than 391 2023-06-27T12:43:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 391 (ObjNum 581 snap 139 seq_num 581) dirty exists 2023-06-27T12:43:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1203: left oid 2 (ObjNum 581 snap 139 seq_num 581) 2023-06-27T12:43:12.664 INFO:tasks.rados.rados.0.smithi138.stdout:1204: finishing write tid 2 to smithi138138488-33 2023-06-27T12:43:12.664 INFO:tasks.rados.rados.0.smithi138.stdout:1204: finishing write tid 3 to smithi138138488-33 2023-06-27T12:43:12.664 INFO:tasks.rados.rados.0.smithi138.stdout:1204: finishing write tid 4 to smithi138138488-33 2023-06-27T12:43:12.664 INFO:tasks.rados.rados.0.smithi138.stdout:1204: finishing write tid 5 to smithi138138488-33 2023-06-27T12:43:12.664 INFO:tasks.rados.rados.0.smithi138.stdout:1204: oid 33 updating version 0 to 120 2023-06-27T12:43:12.664 INFO:tasks.rados.rados.0.smithi138.stdout:1204: oid 33 version 120 is already newer than 120 2023-06-27T12:43:12.664 INFO:tasks.rados.rados.0.smithi138.stdout:1204: oid 33 version 120 is already newer than 120 2023-06-27T12:43:12.664 INFO:tasks.rados.rados.0.smithi138.stdout:1204: oid 33 version 120 is already newer than 116 2023-06-27T12:43:12.665 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 120 (ObjNum 582 snap 139 seq_num 582) dirty exists 2023-06-27T12:43:12.665 INFO:tasks.rados.rados.0.smithi138.stdout:1204: left oid 33 (ObjNum 582 snap 139 seq_num 582) 2023-06-27T12:43:12.665 INFO:tasks.rados.rados.0.smithi138.stdout:1207: expect (ObjNum 578 snap 138 seq_num 578) 2023-06-27T12:43:12.878 INFO:tasks.rados.rados.0.smithi138.stdout:1206: expect (ObjNum 533 snap 122 seq_num 533) 2023-06-27T12:43:13.032 INFO:tasks.rados.rados.0.smithi138.stdout:1209: expect (ObjNum 548 snap 130 seq_num 548) 2023-06-27T12:43:13.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:13.356 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:13.351+0000 7f0da9bac700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:13.359 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:13.356+0000 7f0da9bac700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:13.405 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:13.407 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:13.408 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:13.466 INFO:tasks.rados.rados.0.smithi138.stdout:1202: done (6 left) 2023-06-27T12:43:13.466 INFO:tasks.rados.rados.0.smithi138.stdout:1203: done (5 left) 2023-06-27T12:43:13.466 INFO:tasks.rados.rados.0.smithi138.stdout:1204: done (4 left) 2023-06-27T12:43:13.467 INFO:tasks.rados.rados.0.smithi138.stdout:1206: done (3 left) 2023-06-27T12:43:13.467 INFO:tasks.rados.rados.0.smithi138.stdout:1207: done (2 left) 2023-06-27T12:43:13.467 INFO:tasks.rados.rados.0.smithi138.stdout:1209: done (1 left) 2023-06-27T12:43:13.467 INFO:tasks.rados.rados.0.smithi138.stdout:1210: done (0 left) 2023-06-27T12:43:13.467 INFO:tasks.rados.rados.0.smithi138.stdout:1211: write (excl) oid 7 current snap is 140 2023-06-27T12:43:13.467 INFO:tasks.rados.rados.0.smithi138.stdout:1211: seq_num 584 ranges {615288=788244,1957837=562523,3121263=559679} 2023-06-27T12:43:13.476 INFO:tasks.rados.rados.0.smithi138.stdout:1211: writing smithi138138488-7 from 615288 to 1403532 tid 1 2023-06-27T12:43:13.485 INFO:tasks.rados.rados.0.smithi138.stdout:1211: writing smithi138138488-7 from 1957837 to 2520360 tid 2 2023-06-27T12:43:13.491 INFO:tasks.rados.rados.0.smithi138.stdout:1211: writing smithi138138488-7 from 3121263 to 3680942 tid 3 2023-06-27T12:43:13.491 INFO:tasks.rados.rados.0.smithi138.stdout:1212: delete oid 36 current snap is 140 2023-06-27T12:43:13.541 INFO:tasks.rados.rados.0.smithi138.stdout:1211: finishing write tid 1 to smithi138138488-7 2023-06-27T12:43:13.551 INFO:tasks.rados.rados.0.smithi138.stdout:1212: done (1 left) 2023-06-27T12:43:13.551 INFO:tasks.rados.rados.0.smithi138.stdout:1213: write (excl) oid 8 current snap is 140 2023-06-27T12:43:13.551 INFO:tasks.rados.rados.0.smithi138.stdout:1213: seq_num 586 ranges {648902=655095,2074801=566192,3176109=672793} 2023-06-27T12:43:13.558 INFO:tasks.rados.rados.0.smithi138.stdout:1213: writing smithi138138488-8 from 648902 to 1303997 tid 1 2023-06-27T12:43:13.566 INFO:tasks.rados.rados.0.smithi138.stdout:1213: writing smithi138138488-8 from 2074801 to 2640993 tid 2 2023-06-27T12:43:13.574 INFO:tasks.rados.rados.0.smithi138.stdout:1213: writing smithi138138488-8 from 3176109 to 3848902 tid 3 2023-06-27T12:43:13.574 INFO:tasks.rados.rados.0.smithi138.stdout:1214: delete oid 14 current snap is 140 2023-06-27T12:43:13.595 INFO:tasks.rados.rados.0.smithi138.stdout:1213: finishing write tid 1 to smithi138138488-8 2023-06-27T12:43:13.636 INFO:tasks.rados.rados.0.smithi138.stdout:1211: finishing write tid 2 to smithi138138488-7 2023-06-27T12:43:13.636 INFO:tasks.rados.rados.0.smithi138.stdout:1211: finishing write tid 3 to smithi138138488-7 2023-06-27T12:43:13.636 INFO:tasks.rados.rados.0.smithi138.stdout:1211: finishing write tid 4 to smithi138138488-7 2023-06-27T12:43:13.636 INFO:tasks.rados.rados.0.smithi138.stdout:1211: finishing write tid 5 to smithi138138488-7 2023-06-27T12:43:13.636 INFO:tasks.rados.rados.0.smithi138.stdout:1211: oid 7 updating version 0 to 164 2023-06-27T12:43:13.637 INFO:tasks.rados.rados.0.smithi138.stdout:1211: oid 7 version 164 is already newer than 161 2023-06-27T12:43:13.637 INFO:tasks.rados.rados.0.smithi138.stdout:1211: oid 7 version 164 is already newer than 164 2023-06-27T12:43:13.637 INFO:tasks.rados.rados.0.smithi138.stdout:1211: oid 7 version 164 is already newer than 164 2023-06-27T12:43:13.637 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 164 (ObjNum 584 snap 140 seq_num 584) dirty exists 2023-06-27T12:43:13.637 INFO:tasks.rados.rados.0.smithi138.stdout:1211: left oid 7 (ObjNum 584 snap 140 seq_num 584) 2023-06-27T12:43:13.647 INFO:tasks.rados.rados.0.smithi138.stdout:1211: done (2 left) 2023-06-27T12:43:13.647 INFO:tasks.rados.rados.0.smithi138.stdout:1214: done (1 left) 2023-06-27T12:43:13.647 INFO:tasks.rados.rados.0.smithi138.stdout:1215: write oid 40 current snap is 140 2023-06-27T12:43:13.647 INFO:tasks.rados.rados.0.smithi138.stdout:1215: seq_num 588 ranges {682516=405593,1791765=569861,2830955=721120} 2023-06-27T12:43:13.653 INFO:tasks.rados.rados.0.smithi138.stdout:1215: writing smithi138138488-40 from 682516 to 1088109 tid 1 2023-06-27T12:43:13.660 INFO:tasks.rados.rados.0.smithi138.stdout:1215: writing smithi138138488-40 from 1791765 to 2361626 tid 2 2023-06-27T12:43:13.667 INFO:tasks.rados.rados.0.smithi138.stdout:1215: writing smithi138138488-40 from 2830955 to 3552075 tid 3 2023-06-27T12:43:13.668 INFO:tasks.rados.rados.0.smithi138.stdout:1216: read oid 42 snap -1 2023-06-27T12:43:13.668 INFO:tasks.rados.rados.0.smithi138.stdout:1216: expect (ObjNum 533 snap 122 seq_num 533) 2023-06-27T12:43:13.668 INFO:tasks.rados.rados.0.smithi138.stdout:1217: read oid 20 snap -1 2023-06-27T12:43:13.668 INFO:tasks.rados.rados.0.smithi138.stdout:1217: expect (ObjNum 558 snap 132 seq_num 558) 2023-06-27T12:43:13.668 INFO:tasks.rados.rados.0.smithi138.stdout:1218: write oid 10 current snap is 140 2023-06-27T12:43:13.668 INFO:tasks.rados.rados.0.smithi138.stdout:1213: finishing write tid 2 to smithi138138488-8 2023-06-27T12:43:13.668 INFO:tasks.rados.rados.0.smithi138.stdout:1218: seq_num 589 ranges {699323=480842,1850247=429872,2858378=532392} 2023-06-27T12:43:13.674 INFO:tasks.rados.rados.0.smithi138.stdout:1218: writing smithi138138488-10 from 699323 to 1180165 tid 1 2023-06-27T12:43:13.681 INFO:tasks.rados.rados.0.smithi138.stdout:1218: writing smithi138138488-10 from 1850247 to 2280119 tid 2 2023-06-27T12:43:13.687 INFO:tasks.rados.rados.0.smithi138.stdout:1218: writing smithi138138488-10 from 2858378 to 3390770 tid 3 2023-06-27T12:43:13.688 INFO:tasks.rados.rados.0.smithi138.stdout:1219: read oid 2 snap 138 2023-06-27T12:43:13.688 INFO:tasks.rados.rados.0.smithi138.stdout:1219: expect deleted 2023-06-27T12:43:13.688 INFO:tasks.rados.rados.0.smithi138.stdout:1220: snap_create 2023-06-27T12:43:13.688 INFO:tasks.rados.rados.0.smithi138.stdout:1213: finishing write tid 3 to smithi138138488-8 2023-06-27T12:43:13.688 INFO:tasks.rados.rados.0.smithi138.stdout:1213: finishing write tid 4 to smithi138138488-8 2023-06-27T12:43:13.688 INFO:tasks.rados.rados.0.smithi138.stdout:1213: finishing write tid 5 to smithi138138488-8 2023-06-27T12:43:13.688 INFO:tasks.rados.rados.0.smithi138.stdout:1213: oid 8 updating version 0 to 89 2023-06-27T12:43:13.689 INFO:tasks.rados.rados.0.smithi138.stdout:1213: oid 8 version 89 is already newer than 89 2023-06-27T12:43:13.689 INFO:tasks.rados.rados.0.smithi138.stdout:1213: oid 8 version 89 is already newer than 85 2023-06-27T12:43:13.689 INFO:tasks.rados.rados.0.smithi138.stdout:1213: oid 8 version 89 is already newer than 89 2023-06-27T12:43:13.689 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 8 v 89 (ObjNum 586 snap 140 seq_num 586) dirty exists 2023-06-27T12:43:13.689 INFO:tasks.rados.rados.0.smithi138.stdout:1213: left oid 8 (ObjNum 586 snap 140 seq_num 586) 2023-06-27T12:43:13.689 INFO:tasks.rados.rados.0.smithi138.stdout:1215: finishing write tid 1 to smithi138138488-40 2023-06-27T12:43:13.694 INFO:tasks.rados.rados.0.smithi138.stdout:1215: finishing write tid 2 to smithi138138488-40 2023-06-27T12:43:13.713 INFO:tasks.rados.rados.0.smithi138.stdout:1217: expect (ObjNum 558 snap 132 seq_num 558) 2023-06-27T12:43:13.949 INFO:tasks.rados.rados.0.smithi138.stdout:1215: finishing write tid 3 to smithi138138488-40 2023-06-27T12:43:13.950 INFO:tasks.rados.rados.0.smithi138.stdout:1215: finishing write tid 4 to smithi138138488-40 2023-06-27T12:43:13.950 INFO:tasks.rados.rados.0.smithi138.stdout:1215: finishing write tid 5 to smithi138138488-40 2023-06-27T12:43:13.950 INFO:tasks.rados.rados.0.smithi138.stdout:1215: oid 40 updating version 0 to 127 2023-06-27T12:43:13.950 INFO:tasks.rados.rados.0.smithi138.stdout:1215: oid 40 version 127 is already newer than 125 2023-06-27T12:43:13.950 INFO:tasks.rados.rados.0.smithi138.stdout:1215: oid 40 version 127 is already newer than 127 2023-06-27T12:43:13.951 INFO:tasks.rados.rados.0.smithi138.stdout:1215: oid 40 version 127 is already newer than 127 2023-06-27T12:43:13.951 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 127 (ObjNum 588 snap 140 seq_num 588) dirty exists 2023-06-27T12:43:13.951 INFO:tasks.rados.rados.0.smithi138.stdout:1215: left oid 40 (ObjNum 588 snap 140 seq_num 588) 2023-06-27T12:43:13.951 INFO:tasks.rados.rados.0.smithi138.stdout:1218: finishing write tid 1 to smithi138138488-10 2023-06-27T12:43:13.951 INFO:tasks.rados.rados.0.smithi138.stdout:1218: finishing write tid 2 to smithi138138488-10 2023-06-27T12:43:13.951 INFO:tasks.rados.rados.0.smithi138.stdout:1218: finishing write tid 3 to smithi138138488-10 2023-06-27T12:43:13.951 INFO:tasks.rados.rados.0.smithi138.stdout:1216: expect (ObjNum 533 snap 122 seq_num 533) 2023-06-27T12:43:14.106 INFO:tasks.rados.rados.0.smithi138.stdout:1218: finishing write tid 4 to smithi138138488-10 2023-06-27T12:43:14.106 INFO:tasks.rados.rados.0.smithi138.stdout:1218: finishing write tid 5 to smithi138138488-10 2023-06-27T12:43:14.106 INFO:tasks.rados.rados.0.smithi138.stdout:1218: oid 10 updating version 0 to 169 2023-06-27T12:43:14.106 INFO:tasks.rados.rados.0.smithi138.stdout:1218: oid 10 version 169 is already newer than 169 2023-06-27T12:43:14.106 INFO:tasks.rados.rados.0.smithi138.stdout:1218: oid 10 version 169 is already newer than 169 2023-06-27T12:43:14.106 INFO:tasks.rados.rados.0.smithi138.stdout:1218: oid 10 version 169 is already newer than 169 2023-06-27T12:43:14.107 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 169 (ObjNum 589 snap 140 seq_num 589) dirty exists 2023-06-27T12:43:14.107 INFO:tasks.rados.rados.0.smithi138.stdout:1218: left oid 10 (ObjNum 589 snap 140 seq_num 589) 2023-06-27T12:43:14.478 INFO:tasks.rados.rados.0.smithi138.stdout:1213: done (6 left) 2023-06-27T12:43:14.478 INFO:tasks.rados.rados.0.smithi138.stdout:1215: done (5 left) 2023-06-27T12:43:14.479 INFO:tasks.rados.rados.0.smithi138.stdout:1216: done (4 left) 2023-06-27T12:43:14.479 INFO:tasks.rados.rados.0.smithi138.stdout:1217: done (3 left) 2023-06-27T12:43:14.479 INFO:tasks.rados.rados.0.smithi138.stdout:1218: done (2 left) 2023-06-27T12:43:14.479 INFO:tasks.rados.rados.0.smithi138.stdout:1219: done (1 left) 2023-06-27T12:43:14.479 INFO:tasks.rados.rados.0.smithi138.stdout:1220: done (0 left) 2023-06-27T12:43:14.479 INFO:tasks.rados.rados.0.smithi138.stdout:1221: delete oid 41 current snap is 141 2023-06-27T12:43:14.504 INFO:tasks.rados.rados.0.smithi138.stdout:1221: done (0 left) 2023-06-27T12:43:14.504 INFO:tasks.rados.rados.0.smithi138.stdout:1222: read oid 43 snap -1 2023-06-27T12:43:14.504 INFO:tasks.rados.rados.0.smithi138.stdout:1222: expect (ObjNum 539 snap 124 seq_num 539) 2023-06-27T12:43:14.504 INFO:tasks.rados.rados.0.smithi138.stdout:1223: delete oid 18 current snap is 141 2023-06-27T12:43:14.550 INFO:tasks.rados.rados.0.smithi138.stdout:1222: expect (ObjNum 539 snap 124 seq_num 539) 2023-06-27T12:43:14.732 INFO:tasks.rados.rados.0.smithi138.stdout:1222: done (1 left) 2023-06-27T12:43:14.732 INFO:tasks.rados.rados.0.smithi138.stdout:1223: done (0 left) 2023-06-27T12:43:14.732 INFO:tasks.rados.rados.0.smithi138.stdout:1224: read oid 18 snap 138 2023-06-27T12:43:14.732 INFO:tasks.rados.rados.0.smithi138.stdout:1224: expect (ObjNum 470 snap 110 seq_num 470) 2023-06-27T12:43:14.732 INFO:tasks.rados.rados.0.smithi138.stdout:1225: write oid 35 current snap is 141 2023-06-27T12:43:14.733 INFO:tasks.rados.rados.0.smithi138.stdout:1225: seq_num 592 ranges {749744=706589,1909340=577199,3224294=482561} 2023-06-27T12:43:14.740 INFO:tasks.rados.rados.0.smithi138.stdout:1225: writing smithi138138488-35 from 749744 to 1456333 tid 1 2023-06-27T12:43:14.745 INFO:tasks.rados.rados.0.smithi138.stdout:1225: writing smithi138138488-35 from 1909340 to 2486539 tid 2 2023-06-27T12:43:14.752 INFO:tasks.rados.rados.0.smithi138.stdout:1225: writing smithi138138488-35 from 3224294 to 3706855 tid 3 2023-06-27T12:43:14.753 INFO:tasks.rados.rados.0.smithi138.stdout:1226: snap_create 2023-06-27T12:43:14.753 INFO:tasks.rados.rados.0.smithi138.stdout:1224: expect (ObjNum 470 snap 110 seq_num 470) 2023-06-27T12:43:14.900 INFO:tasks.rados.rados.0.smithi138.stdout:1225: finishing write tid 1 to smithi138138488-35 2023-06-27T12:43:14.900 INFO:tasks.rados.rados.0.smithi138.stdout:1225: finishing write tid 2 to smithi138138488-35 2023-06-27T12:43:14.900 INFO:tasks.rados.rados.0.smithi138.stdout:1225: finishing write tid 3 to smithi138138488-35 2023-06-27T12:43:14.901 INFO:tasks.rados.rados.0.smithi138.stdout:1225: finishing write tid 4 to smithi138138488-35 2023-06-27T12:43:14.901 INFO:tasks.rados.rados.0.smithi138.stdout:1225: finishing write tid 5 to smithi138138488-35 2023-06-27T12:43:14.901 INFO:tasks.rados.rados.0.smithi138.stdout:1225: oid 35 updating version 0 to 398 2023-06-27T12:43:14.901 INFO:tasks.rados.rados.0.smithi138.stdout:1225: oid 35 updating version 398 to 400 2023-06-27T12:43:14.901 INFO:tasks.rados.rados.0.smithi138.stdout:1225: oid 35 version 400 is already newer than 400 2023-06-27T12:43:14.901 INFO:tasks.rados.rados.0.smithi138.stdout:1225: oid 35 version 400 is already newer than 399 2023-06-27T12:43:14.901 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 400 (ObjNum 592 snap 141 seq_num 592) dirty exists 2023-06-27T12:43:14.902 INFO:tasks.rados.rados.0.smithi138.stdout:1225: left oid 35 (ObjNum 592 snap 141 seq_num 592) 2023-06-27T12:43:14.928 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~160s 2023-06-27T12:43:15.409 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:15.443 INFO:tasks.rados.rados.0.smithi138.stdout:1224: done (2 left) 2023-06-27T12:43:15.443 INFO:tasks.rados.rados.0.smithi138.stdout:1225: done (1 left) 2023-06-27T12:43:15.443 INFO:tasks.rados.rados.0.smithi138.stdout:1226: done (0 left) 2023-06-27T12:43:15.444 INFO:tasks.rados.rados.0.smithi138.stdout:1227: read oid 28 snap 139 2023-06-27T12:43:15.444 INFO:tasks.rados.rados.0.smithi138.stdout:1227: expect deleted 2023-06-27T12:43:15.444 INFO:tasks.rados.rados.0.smithi138.stdout:1228: snap_remove snap 141 2023-06-27T12:43:15.543 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:15.538+0000 7f00a517f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:15.546 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:15.542+0000 7f00a517f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:15.592 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:15.595 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:15.595 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:16.396 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:16.397 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:16.397 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:16.444 INFO:tasks.rados.rados.0.smithi138.stdout:1228: done (1 left) 2023-06-27T12:43:16.444 INFO:tasks.rados.rados.0.smithi138.stdout:1229: read oid 46 snap -1 2023-06-27T12:43:16.444 INFO:tasks.rados.rados.0.smithi138.stdout:1229: expect deleted 2023-06-27T12:43:16.444 INFO:tasks.rados.rados.0.smithi138.stdout:1230: read oid 14 snap -1 2023-06-27T12:43:16.445 INFO:tasks.rados.rados.0.smithi138.stdout:1230: expect deleted 2023-06-27T12:43:16.445 INFO:tasks.rados.rados.0.smithi138.stdout:1231: read oid 29 snap 140 2023-06-27T12:43:16.445 INFO:tasks.rados.rados.0.smithi138.stdout:1231: expect (ObjNum 484 snap 113 seq_num 484) 2023-06-27T12:43:16.445 INFO:tasks.rados.rados.0.smithi138.stdout:1232: delete oid 1 current snap is 142 2023-06-27T12:43:16.530 INFO:tasks.rados.rados.0.smithi138.stdout:1227: done (4 left) 2023-06-27T12:43:16.530 INFO:tasks.rados.rados.0.smithi138.stdout:1229: done (3 left) 2023-06-27T12:43:16.530 INFO:tasks.rados.rados.0.smithi138.stdout:1230: done (2 left) 2023-06-27T12:43:16.530 INFO:tasks.rados.rados.0.smithi138.stdout:1232: done (1 left) 2023-06-27T12:43:16.531 INFO:tasks.rados.rados.0.smithi138.stdout:1233: write oid 9 current snap is 142 2023-06-27T12:43:16.531 INFO:tasks.rados.rados.0.smithi138.stdout:1233: seq_num 594 ranges {783358=573440,2026304=580868,3279140=621458} 2023-06-27T12:43:16.537 INFO:tasks.rados.rados.0.smithi138.stdout:1233: writing smithi138138488-9 from 783358 to 1356798 tid 1 2023-06-27T12:43:16.544 INFO:tasks.rados.rados.0.smithi138.stdout:1233: writing smithi138138488-9 from 2026304 to 2607172 tid 2 2023-06-27T12:43:16.552 INFO:tasks.rados.rados.0.smithi138.stdout:1233: writing smithi138138488-9 from 3279140 to 3900598 tid 3 2023-06-27T12:43:16.552 INFO:tasks.rados.rados.0.smithi138.stdout:1234: write oid 39 current snap is 142 2023-06-27T12:43:16.552 INFO:tasks.rados.rados.0.smithi138.stdout:1231: expect (ObjNum 484 snap 113 seq_num 484) 2023-06-27T12:43:16.556 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:16.547+0000 7f676aafc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:16.559 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:16.555+0000 7f676aafc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:16.650 INFO:tasks.rados.rados.0.smithi138.stdout:1233: finishing write tid 1 to smithi138138488-9 2023-06-27T12:43:16.650 INFO:tasks.rados.rados.0.smithi138.stdout:1234: seq_num 595 ranges {400165=648689,1684786=315379} 2023-06-27T12:43:16.656 INFO:tasks.rados.rados.0.smithi138.stdout:1234: writing smithi138138488-39 from 400165 to 1048854 tid 1 2023-06-27T12:43:16.661 INFO:tasks.rados.rados.0.smithi138.stdout:1234: writing smithi138138488-39 from 1684786 to 2000165 tid 2 2023-06-27T12:43:16.662 INFO:tasks.rados.rados.0.smithi138.stdout:1231: done (2 left) 2023-06-27T12:43:16.662 INFO:tasks.rados.rados.0.smithi138.stdout:1235: snap_create 2023-06-27T12:43:16.662 INFO:tasks.rados.rados.0.smithi138.stdout:1233: finishing write tid 2 to smithi138138488-9 2023-06-27T12:43:16.662 INFO:tasks.rados.rados.0.smithi138.stdout:1233: finishing write tid 3 to smithi138138488-9 2023-06-27T12:43:16.662 INFO:tasks.rados.rados.0.smithi138.stdout:1233: finishing write tid 4 to smithi138138488-9 2023-06-27T12:43:16.662 INFO:tasks.rados.rados.0.smithi138.stdout:1233: finishing write tid 5 to smithi138138488-9 2023-06-27T12:43:16.663 INFO:tasks.rados.rados.0.smithi138.stdout:1233: oid 9 updating version 0 to 356 2023-06-27T12:43:16.663 INFO:tasks.rados.rados.0.smithi138.stdout:1233: oid 9 version 356 is already newer than 352 2023-06-27T12:43:16.663 INFO:tasks.rados.rados.0.smithi138.stdout:1233: oid 9 version 356 is already newer than 356 2023-06-27T12:43:16.663 INFO:tasks.rados.rados.0.smithi138.stdout:1233: oid 9 version 356 is already newer than 356 2023-06-27T12:43:16.663 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 356 (ObjNum 594 snap 142 seq_num 594) dirty exists 2023-06-27T12:43:16.663 INFO:tasks.rados.rados.0.smithi138.stdout:1233: left oid 9 (ObjNum 594 snap 142 seq_num 594) 2023-06-27T12:43:16.710 INFO:tasks.rados.rados.0.smithi138.stdout:1234: finishing write tid 1 to smithi138138488-39 2023-06-27T12:43:16.751 INFO:tasks.rados.rados.0.smithi138.stdout:1234: finishing write tid 2 to smithi138138488-39 2023-06-27T12:43:16.751 INFO:tasks.rados.rados.0.smithi138.stdout:1234: finishing write tid 3 to smithi138138488-39 2023-06-27T12:43:16.751 INFO:tasks.rados.rados.0.smithi138.stdout:1234: finishing write tid 4 to smithi138138488-39 2023-06-27T12:43:16.751 INFO:tasks.rados.rados.0.smithi138.stdout:1234: oid 39 updating version 0 to 200 2023-06-27T12:43:16.751 INFO:tasks.rados.rados.0.smithi138.stdout:1234: oid 39 version 200 is already newer than 200 2023-06-27T12:43:16.752 INFO:tasks.rados.rados.0.smithi138.stdout:1234: oid 39 version 200 is already newer than 200 2023-06-27T12:43:16.752 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 39 v 200 (ObjNum 595 snap 142 seq_num 595) dirty exists 2023-06-27T12:43:16.752 INFO:tasks.rados.rados.0.smithi138.stdout:1234: left oid 39 (ObjNum 595 snap 142 seq_num 595) 2023-06-27T12:43:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:16.775 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:16.787 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2023-06-27T12:43:16.439001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":114828625,"num_objects":301,"num_object_clones":254,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1809,"num_read_kb":1320192,"num_write":1615,"num_write_kb":472472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2982,"ondisk_log_size":2982,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":930744,"kb_used_data":794776,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376556616,"statfs":{"total":386547056640,"available":385593974784,"internally_reserved":0,"allocated":813850624,"data_stored":923136185,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4369754,"num_objects":12,"num_object_clones":11,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":52029,"num_write":51,"num_write_kb":17539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.979647"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":717,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.476655+0000","last_change":"2023-06-27T12:43:12.501252+0000","last_active":"2023-06-27T12:43:13.476655+0000","last_peered":"2023-06-27T12:43:13.476655+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:13.476655+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"298'180","reported_seq":940,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.699329+0000","last_change":"2023-06-27T12:43:12.486527+0000","last_active":"2023-06-27T12:43:13.699329+0000","last_peered":"2023-06-27T12:43:13.699329+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:13.699329+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14504771,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":157390,"num_write":129,"num_write_kb":38261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"297'248","reported_seq":938,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.499791+0000","last_change":"2023-06-27T12:43:12.577376+0000","last_active":"2023-06-27T12:43:13.499791+0000","last_peered":"2023-06-27T12:43:13.499791+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:13.499791+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9361763,"num_objects":29,"num_object_clones":24,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":198,"num_read_kb":121762,"num_write":165,"num_write_kb":47622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":29}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"300'127","reported_seq":876,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.762691+0000","last_change":"2023-06-27T12:43:12.555821+0000","last_active":"2023-06-27T12:43:13.762691+0000","last_peered":"2023-06-27T12:43:13.762691+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:13.762691+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10622321,"num_objects":21,"num_object_clones":17,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":138,"num_read_kb":96599,"num_write":98,"num_write_kb":28260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":21}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"300'89","reported_seq":437,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.596369+0000","last_change":"2023-06-27T12:43:12.501720+0000","last_active":"2023-06-27T12:43:13.596369+0000","last_peered":"2023-06-27T12:43:13.596369+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:13.596369+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7378471,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":50182,"num_write":49,"num_write_kb":15136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"299'371","reported_seq":967,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-06-27T12:43:13.473065+0000","last_change":"2023-06-27T12:43:12.502300+0000","last_active":"2023-06-27T12:43:13.473065+0000","last_peered":"2023-06-27T12:43:13.473065+0000","last_clean":"2023-06-27T12:43:13.473065+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:13.473065+0000","last_undegraded":"2023-06-27T12:43:13.473065+0000","last_fullsized":"2023-06-27T12:43:13.473065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10823972,"num_objects":41,"num_object_clones":35,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":291,"num_read_kb":185367,"num_write":220,"num_write_kb":63975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"300'169","reported_seq":776,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-06-27T12:43:13.748577+0000","last_change":"2023-06-27T12:43:12.507520+0000","last_active":"2023-06-27T12:43:13.748577+0000","last_peered":"2023-06-27T12:43:13.748577+0000","last_clean":"2023-06-27T12:43:13.748577+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:13.748577+0000","last_undegraded":"2023-06-27T12:43:13.748577+0000","last_fullsized":"2023-06-27T12:43:13.748577+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7714611,"num_objects":20,"num_object_clones":17,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":114,"num_read_kb":78358,"num_write":100,"num_write_kb":29518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":754,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.517744+0000","last_change":"2023-06-27T12:43:12.488526+0000","last_active":"2023-06-27T12:43:13.517744+0000","last_peered":"2023-06-27T12:43:13.517744+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:13.517744+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":498,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-06-27T12:43:13.677377+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:13.677377+0000","last_peered":"2023-06-27T12:43:13.677377+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:13.677377+0000","last_undegraded":"2023-06-27T12:43:13.677377+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":668,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-06-27T12:43:13.512909+0000","last_change":"2023-06-27T12:43:12.516327+0000","last_active":"2023-06-27T12:43:13.512909+0000","last_peered":"2023-06-27T12:43:13.512909+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:13.512909+0000","last_undegraded":"2023-06-27T12:43:13.512909+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":479,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-06-27T12:43:13.480922+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:13.480922+0000","last_peered":"2023-06-27T12:43:13.480922+0000","last_clean":"2023-06-27T12:43:13.480922+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:13.480922+0000","last_undegraded":"2023-06-27T12:43:13.480922+0000","last_fullsized":"2023-06-27T12:43:13.480922+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"291'167","reported_seq":864,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.686449+0000","last_change":"2023-06-27T12:43:12.488624+0000","last_active":"2023-06-27T12:43:13.686449+0000","last_peered":"2023-06-27T12:43:13.686449+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:13.686449+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420472,"num_objects":19,"num_object_clones":14,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":70,"num_read_kb":66416,"num_write":106,"num_write_kb":29767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":466,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-06-27T12:43:13.517891+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:13.517891+0000","last_peered":"2023-06-27T12:43:13.517891+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:13.517891+0000","last_undegraded":"2023-06-27T12:43:13.517891+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"298'391","reported_seq":525,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.680833+0000","last_change":"2023-06-27T12:43:12.562024+0000","last_active":"2023-06-27T12:43:13.680833+0000","last_peered":"2023-06-27T12:43:13.680833+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:13.680833+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10202113,"num_objects":19,"num_object_clones":17,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":52,"num_read_kb":29631,"num_write":99,"num_write_kb":30703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":463,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-06-27T12:43:13.517724+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:13.517724+0000","last_peered":"2023-06-27T12:43:13.517724+0000","last_clean":"2023-06-27T12:43:13.517724+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:13.517724+0000","last_undegraded":"2023-06-27T12:43:13.517724+0000","last_fullsized":"2023-06-27T12:43:13.517724+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":231,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-06-27T12:43:13.475927+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:13.475927+0000","last_peered":"2023-06-27T12:43:13.475927+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:13.475927+0000","last_undegraded":"2023-06-27T12:43:13.475927+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"288'347","reported_seq":1064,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.516297+0000","last_change":"2023-06-27T12:43:12.490506+0000","last_active":"2023-06-27T12:43:13.516297+0000","last_peered":"2023-06-27T12:43:13.516297+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:13.516297+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6689384,"num_objects":45,"num_object_clones":39,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":285,"num_read_kb":173900,"num_write":252,"num_write_kb":73394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":45}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":479,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-06-27T12:43:13.472453+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:13.472453+0000","last_peered":"2023-06-27T12:43:13.472453+0000","last_clean":"2023-06-27T12:43:13.472453+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:13.472453+0000","last_undegraded":"2023-06-27T12:43:13.472453+0000","last_fullsized":"2023-06-27T12:43:13.472453+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"293'196","reported_seq":444,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.515542+0000","last_change":"2023-06-27T12:43:12.517147+0000","last_active":"2023-06-27T12:43:13.515542+0000","last_peered":"2023-06-27T12:43:13.515542+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:13.515542+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857454,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":85,"num_read_kb":47366,"num_write":59,"num_write_kb":16910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-06-27T12:43:13.679932+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:13.679932+0000","last_peered":"2023-06-27T12:43:13.679932+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:13.679932+0000","last_undegraded":"2023-06-27T12:43:13.679932+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"286'173","reported_seq":893,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.683119+0000","last_change":"2023-06-27T12:43:12.486416+0000","last_active":"2023-06-27T12:43:13.683119+0000","last_peered":"2023-06-27T12:43:13.683119+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:13.683119+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6134753,"num_objects":23,"num_object_clones":19,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":93,"num_read_kb":105200,"num_write":114,"num_write_kb":33633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"297'109","reported_seq":429,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.501521+0000","last_change":"2023-06-27T12:43:12.536407+0000","last_active":"2023-06-27T12:43:13.501521+0000","last_peered":"2023-06-27T12:43:13.501521+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:13.501521+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5815420,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":48531,"num_write":39,"num_write_kb":11125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":498,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-06-27T12:43:13.685128+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:13.685128+0000","last_peered":"2023-06-27T12:43:13.685128+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:13.685128+0000","last_undegraded":"2023-06-27T12:43:13.685128+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":463,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-06-27T12:43:13.502816+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:13.502816+0000","last_peered":"2023-06-27T12:43:13.502816+0000","last_clean":"2023-06-27T12:43:13.502816+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:13.502816+0000","last_undegraded":"2023-06-27T12:43:13.502816+0000","last_fullsized":"2023-06-27T12:43:13.502816+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"297'214","reported_seq":909,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:13.681202+0000","last_change":"2023-06-27T12:43:12.561329+0000","last_active":"2023-06-27T12:43:13.681202+0000","last_peered":"2023-06-27T12:43:13.681202+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:13.681202+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857454,"num_objects":24,"num_object_clones":21,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":108,"num_read_kb":112357,"num_write":116,"num_write_kb":34264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":24}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":114828625,"num_objects":301,"num_object_clones":254,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1809,"num_read_kb":1320192,"num_write":1615,"num_write_kb":472472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2885,"ondisk_log_size":2885,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":240596,"kb_used_data":205580,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94131244,"statfs":{"total":96636764160,"available":96390393856,"internally_reserved":0,"allocated":210513920,"data_stored":239045166,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":242384,"kb_used_data":207368,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94129456,"statfs":{"total":96636764160,"available":96388562944,"internally_reserved":0,"allocated":212344832,"data_stored":239121596,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":266484,"kb_used_data":231468,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94105356,"statfs":{"total":96636764160,"available":96363884544,"internally_reserved":0,"allocated":237023232,"data_stored":267929871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:16.789 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.c 3.8 3.3 3.1 3.4 2023-06-27T12:43:16.923 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:16.908+0000 7f72612f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:16.926 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:16.922+0000 7f72612f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:17.121 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c,3.f] on osd.0 to force-recovery; instructing pg(s) [3.1,3.8] on osd.1 to force-recovery; instructing pg(s) [3.3,3.4,3.e] on osd.2 to force-recovery; 2023-06-27T12:43:17.122 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:17.489 INFO:tasks.rados.rados.0.smithi138.stdout:1233: done (2 left) 2023-06-27T12:43:17.489 INFO:tasks.rados.rados.0.smithi138.stdout:1234: done (1 left) 2023-06-27T12:43:17.489 INFO:tasks.rados.rados.0.smithi138.stdout:1235: done (0 left) 2023-06-27T12:43:17.489 INFO:tasks.rados.rados.0.smithi138.stdout:1236: write oid 32 current snap is 143 2023-06-27T12:43:17.490 INFO:tasks.rados.rados.0.smithi138.stdout:1236: seq_num 596 ranges {416972=723938,1743268=273704} 2023-06-27T12:43:17.495 INFO:tasks.rados.rados.0.smithi138.stdout:1236: writing smithi138138488-32 from 416972 to 1140910 tid 1 2023-06-27T12:43:17.500 INFO:tasks.rados.rados.0.smithi138.stdout:1236: writing smithi138138488-32 from 1743268 to 2016972 tid 2 2023-06-27T12:43:17.500 INFO:tasks.rados.rados.0.smithi138.stdout:1237: delete oid 34 current snap is 143 2023-06-27T12:43:17.520 INFO:tasks.rados.rados.0.smithi138.stdout:1237: done (1 left) 2023-06-27T12:43:17.521 INFO:tasks.rados.rados.0.smithi138.stdout:1238: read oid 20 snap -1 2023-06-27T12:43:17.521 INFO:tasks.rados.rados.0.smithi138.stdout:1238: expect (ObjNum 558 snap 132 seq_num 558) 2023-06-27T12:43:17.521 INFO:tasks.rados.rados.0.smithi138.stdout:1239: delete oid 36 current snap is 143 2023-06-27T12:43:17.523 INFO:tasks.rados.rados.0.smithi138.stdout:1239: done (2 left) 2023-06-27T12:43:17.523 INFO:tasks.rados.rados.0.smithi138.stdout:1240: snap_create 2023-06-27T12:43:17.554 INFO:tasks.rados.rados.0.smithi138.stdout:1238: expect (ObjNum 558 snap 132 seq_num 558) 2023-06-27T12:43:17.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:17.724 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:17.719+0000 7fea0da24700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:17.726 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:17.723+0000 7fea0da24700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:17.771 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:17.775 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:17.775 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:17.797 INFO:tasks.rados.rados.0.smithi138.stdout:1236: finishing write tid 1 to smithi138138488-32 2023-06-27T12:43:17.798 INFO:tasks.rados.rados.0.smithi138.stdout:1236: finishing write tid 2 to smithi138138488-32 2023-06-27T12:43:17.798 INFO:tasks.rados.rados.0.smithi138.stdout:1236: finishing write tid 3 to smithi138138488-32 2023-06-27T12:43:17.798 INFO:tasks.rados.rados.0.smithi138.stdout:1236: finishing write tid 4 to smithi138138488-32 2023-06-27T12:43:17.798 INFO:tasks.rados.rados.0.smithi138.stdout:1236: oid 32 updating version 0 to 404 2023-06-27T12:43:17.798 INFO:tasks.rados.rados.0.smithi138.stdout:1236: oid 32 version 404 is already newer than 404 2023-06-27T12:43:17.798 INFO:tasks.rados.rados.0.smithi138.stdout:1236: oid 32 version 404 is already newer than 404 2023-06-27T12:43:17.798 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 404 (ObjNum 596 snap 143 seq_num 596) dirty exists 2023-06-27T12:43:17.799 INFO:tasks.rados.rados.0.smithi138.stdout:1236: left oid 32 (ObjNum 596 snap 143 seq_num 596) 2023-06-27T12:43:18.493 INFO:tasks.rados.rados.0.smithi138.stdout:1236: done (2 left) 2023-06-27T12:43:18.493 INFO:tasks.rados.rados.0.smithi138.stdout:1238: done (1 left) 2023-06-27T12:43:18.493 INFO:tasks.rados.rados.0.smithi138.stdout:1240: done (0 left) 2023-06-27T12:43:18.493 INFO:tasks.rados.rados.0.smithi138.stdout:1241: delete oid 15 current snap is 144 2023-06-27T12:43:18.559 INFO:tasks.rados.rados.0.smithi138.stdout:1241: done (0 left) 2023-06-27T12:43:18.560 INFO:tasks.rados.rados.0.smithi138.stdout:1242: read oid 32 snap -1 2023-06-27T12:43:18.560 INFO:tasks.rados.rados.0.smithi138.stdout:1242: expect (ObjNum 596 snap 143 seq_num 596) 2023-06-27T12:43:18.560 INFO:tasks.rados.rados.0.smithi138.stdout:1243: read oid 49 snap -1 2023-06-27T12:43:18.560 INFO:tasks.rados.rados.0.smithi138.stdout:1243: expect deleted 2023-06-27T12:43:18.560 INFO:tasks.rados.rados.0.smithi138.stdout:1244: read oid 37 snap 138 2023-06-27T12:43:18.560 INFO:tasks.rados.rados.0.smithi138.stdout:1244: expect deleted 2023-06-27T12:43:18.560 INFO:tasks.rados.rados.0.smithi138.stdout:1245: write (excl) oid 28 current snap is 144 2023-06-27T12:43:18.561 INFO:tasks.rados.rados.0.smithi138.stdout:1245: seq_num 600 ranges {484200=624934,1860843=223357} 2023-06-27T12:43:18.566 INFO:tasks.rados.rados.0.smithi138.stdout:1245: writing smithi138138488-28 from 484200 to 1109134 tid 1 2023-06-27T12:43:18.571 INFO:tasks.rados.rados.0.smithi138.stdout:1245: writing smithi138138488-28 from 1860843 to 2084200 tid 2 2023-06-27T12:43:18.572 INFO:tasks.rados.rados.0.smithi138.stdout:1246: read oid 34 snap -1 2023-06-27T12:43:18.572 INFO:tasks.rados.rados.0.smithi138.stdout:1246: expect deleted 2023-06-27T12:43:18.572 INFO:tasks.rados.rados.0.smithi138.stdout:1247: write oid 40 current snap is 144 2023-06-27T12:43:18.572 INFO:tasks.rados.rados.0.smithi138.stdout:1247: seq_num 601 ranges {501007=416536,1635678=465329} 2023-06-27T12:43:18.578 INFO:tasks.rados.rados.0.smithi138.stdout:1247: writing smithi138138488-40 from 501007 to 917543 tid 1 2023-06-27T12:43:18.586 INFO:tasks.rados.rados.0.smithi138.stdout:1247: writing smithi138138488-40 from 1635678 to 2101007 tid 2 2023-06-27T12:43:18.586 INFO:tasks.rados.rados.0.smithi138.stdout:1248: write (excl) oid 48 current snap is 144 2023-06-27T12:43:18.587 INFO:tasks.rados.rados.0.smithi138.stdout:1248: seq_num 602 ranges {517814=491785,1577807=540007} 2023-06-27T12:43:18.593 INFO:tasks.rados.rados.0.smithi138.stdout:1248: writing smithi138138488-48 from 517814 to 1009599 tid 1 2023-06-27T12:43:18.600 INFO:tasks.rados.rados.0.smithi138.stdout:1248: writing smithi138138488-48 from 1577807 to 2117814 tid 2 2023-06-27T12:43:18.600 INFO:tasks.rados.rados.0.smithi138.stdout:1249: snap_remove snap 143 2023-06-27T12:43:19.499 INFO:tasks.rados.rados.0.smithi138.stdout:1249: done (7 left) 2023-06-27T12:43:19.499 INFO:tasks.rados.rados.0.smithi138.stdout:1250: read oid 15 snap -1 2023-06-27T12:43:19.499 INFO:tasks.rados.rados.0.smithi138.stdout:1250: expect deleted 2023-06-27T12:43:19.500 INFO:tasks.rados.rados.0.smithi138.stdout:1251: write oid 27 current snap is 144 2023-06-27T12:43:19.500 INFO:tasks.rados.rados.0.smithi138.stdout:1251: seq_num 603 ranges {534621=567034,1636289=498332} 2023-06-27T12:43:19.505 INFO:tasks.rados.rados.0.smithi138.stdout:1251: writing smithi138138488-27 from 534621 to 1101655 tid 1 2023-06-27T12:43:19.511 INFO:tasks.rados.rados.0.smithi138.stdout:1251: writing smithi138138488-27 from 1636289 to 2134621 tid 2 2023-06-27T12:43:19.511 INFO:tasks.rados.rados.0.smithi138.stdout:1252: read oid 21 snap -1 2023-06-27T12:43:19.512 INFO:tasks.rados.rados.0.smithi138.stdout:1252: expect deleted 2023-06-27T12:43:19.512 INFO:tasks.rados.rados.0.smithi138.stdout:1253: read oid 16 snap -1 2023-06-27T12:43:19.512 INFO:tasks.rados.rados.0.smithi138.stdout:1253: expect (ObjNum 562 snap 132 seq_num 562) 2023-06-27T12:43:19.512 INFO:tasks.rados.rados.0.smithi138.stdout:1254: snap_remove snap 142 2023-06-27T12:43:19.777 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:19.898+0000 7f1610068700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:19.902+0000 7f1610068700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:19.952 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:19.955 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:19.956 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:20.436 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~165s 2023-06-27T12:43:20.504 INFO:tasks.rados.rados.0.smithi138.stdout:1254: done (11 left) 2023-06-27T12:43:20.504 INFO:tasks.rados.rados.0.smithi138.stdout:1255: read oid 6 snap -1 2023-06-27T12:43:20.504 INFO:tasks.rados.rados.0.smithi138.stdout:1255: expect (ObjNum 524 snap 121 seq_num 524) 2023-06-27T12:43:20.505 INFO:tasks.rados.rados.0.smithi138.stdout:1256: snap_remove snap 139 2023-06-27T12:43:21.507 INFO:tasks.rados.rados.0.smithi138.stdout:1256: done (12 left) 2023-06-27T12:43:21.507 INFO:tasks.rados.rados.0.smithi138.stdout:1257: snap_remove snap 133 2023-06-27T12:43:21.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:22.087 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:22.082+0000 7f1ce79e8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:22.090 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:22.086+0000 7f1ce79e8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:22.134 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:22.134 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:22.134 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:22.137 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:22.139 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:22.139 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:22.261 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:22.254+0000 7f5b2e6c2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:22.264 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:22.260+0000 7f5b2e6c2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:22.458 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:22.460 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:22.472 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2023-06-27T12:43:21.978571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":101164534,"num_objects":310,"num_object_clones":263,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1841,"num_read_kb":1348534,"num_write":1645,"num_write_kb":480552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3027,"ondisk_log_size":3027,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":947960,"kb_used_data":811992,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376539400,"statfs":{"total":386547056640,"available":385576345600,"internally_reserved":0,"allocated":831479808,"data_stored":948071073,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13664091,"num_objects":9,"num_object_clones":9,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":32,"num_read_kb":28342,"num_write":30,"num_write_kb":8080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001051"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":728,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.505655+0000","last_change":"2023-06-27T12:43:16.473505+0000","last_active":"2023-06-27T12:43:18.505655+0000","last_peered":"2023-06-27T12:43:18.505655+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:18.505655+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"298'180","reported_seq":953,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.507395+0000","last_change":"2023-06-27T12:43:16.464413+0000","last_active":"2023-06-27T12:43:18.507395+0000","last_peered":"2023-06-27T12:43:18.507395+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:18.507395+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14504771,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":163043,"num_write":129,"num_write_kb":38261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"301'254","reported_seq":954,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.590725+0000","last_change":"2023-06-27T12:43:16.484463+0000","last_active":"2023-06-27T12:43:18.590725+0000","last_peered":"2023-06-27T12:43:18.590725+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:18.590725+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5462473,"num_objects":30,"num_object_clones":25,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":206,"num_read_kb":131103,"num_write":165,"num_write_kb":47622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"305'131","reported_seq":891,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.655276+0000","last_change":"2023-06-27T12:43:16.480107+0000","last_active":"2023-06-27T12:43:18.655276+0000","last_peered":"2023-06-27T12:43:18.655276+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:18.655276+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840812,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":139,"num_read_kb":96600,"num_write":102,"num_write_kb":29122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"300'89","reported_seq":449,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.501094+0000","last_change":"2023-06-27T12:43:16.497381+0000","last_active":"2023-06-27T12:43:18.501094+0000","last_peered":"2023-06-27T12:43:18.501094+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:18.501094+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3848902,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":88,"num_read_kb":50182,"num_write":49,"num_write_kb":15136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"299'371","reported_seq":979,"reported_epoch":305,"state":"active+clean","last_fresh":"2023-06-27T12:43:18.506827+0000","last_change":"2023-06-27T12:43:16.497943+0000","last_active":"2023-06-27T12:43:18.506827+0000","last_peered":"2023-06-27T12:43:18.506827+0000","last_clean":"2023-06-27T12:43:18.506827+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:18.506827+0000","last_undegraded":"2023-06-27T12:43:18.506827+0000","last_fullsized":"2023-06-27T12:43:18.506827+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10823972,"num_objects":41,"num_object_clones":35,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":291,"num_read_kb":185367,"num_write":220,"num_write_kb":63975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"305'172","reported_seq":791,"reported_epoch":305,"state":"active+clean","last_fresh":"2023-06-27T12:43:18.663031+0000","last_change":"2023-06-27T12:43:16.481137+0000","last_active":"2023-06-27T12:43:18.663031+0000","last_peered":"2023-06-27T12:43:18.663031+0000","last_clean":"2023-06-27T12:43:18.663031+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:18.663031+0000","last_undegraded":"2023-06-27T12:43:18.663031+0000","last_fullsized":"2023-06-27T12:43:18.663031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9798811,"num_objects":20,"num_object_clones":17,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":78359,"num_write":104,"num_write_kb":30348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":765,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.575854+0000","last_change":"2023-06-27T12:43:16.490070+0000","last_active":"2023-06-27T12:43:18.575854+0000","last_peered":"2023-06-27T12:43:18.575854+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:18.575854+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":508,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-06-27T12:43:18.496746+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:18.496746+0000","last_peered":"2023-06-27T12:43:18.496746+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:18.496746+0000","last_undegraded":"2023-06-27T12:43:18.496746+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":679,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-06-27T12:43:18.573218+0000","last_change":"2023-06-27T12:43:16.484070+0000","last_active":"2023-06-27T12:43:18.573218+0000","last_peered":"2023-06-27T12:43:18.573218+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:18.573218+0000","last_undegraded":"2023-06-27T12:43:18.573218+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":489,"reported_epoch":305,"state":"active+clean","last_fresh":"2023-06-27T12:43:18.503974+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:18.503974+0000","last_peered":"2023-06-27T12:43:18.503974+0000","last_clean":"2023-06-27T12:43:18.503974+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:18.503974+0000","last_undegraded":"2023-06-27T12:43:18.503974+0000","last_fullsized":"2023-06-27T12:43:18.503974+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"291'167","reported_seq":875,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.504956+0000","last_change":"2023-06-27T12:43:16.465510+0000","last_active":"2023-06-27T12:43:18.504956+0000","last_peered":"2023-06-27T12:43:18.504956+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:18.504956+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420472,"num_objects":19,"num_object_clones":14,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":70,"num_read_kb":66416,"num_write":106,"num_write_kb":29767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":476,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-06-27T12:43:18.589078+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:18.589078+0000","last_peered":"2023-06-27T12:43:18.589078+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:18.589078+0000","last_undegraded":"2023-06-27T12:43:18.589078+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"304'404","reported_seq":547,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.562969+0000","last_change":"2023-06-27T12:43:16.466063+0000","last_active":"2023-06-27T12:43:18.562969+0000","last_peered":"2023-06-27T12:43:18.562969+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:18.562969+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9025656,"num_objects":21,"num_object_clones":19,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":56,"num_read_kb":33573,"num_write":108,"num_write_kb":33405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":473,"reported_epoch":305,"state":"active+clean","last_fresh":"2023-06-27T12:43:18.584347+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:18.584347+0000","last_peered":"2023-06-27T12:43:18.584347+0000","last_clean":"2023-06-27T12:43:18.584347+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:18.584347+0000","last_undegraded":"2023-06-27T12:43:18.584347+0000","last_fullsized":"2023-06-27T12:43:18.584347+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":241,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-06-27T12:43:18.506496+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:18.506496+0000","last_peered":"2023-06-27T12:43:18.506496+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:18.506496+0000","last_undegraded":"2023-06-27T12:43:18.506496+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"303'356","reported_seq":1081,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.577069+0000","last_change":"2023-06-27T12:43:16.474420+0000","last_active":"2023-06-27T12:43:18.577069+0000","last_peered":"2023-06-27T12:43:18.577069+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:18.577069+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3983358,"num_objects":47,"num_object_clones":41,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":287,"num_read_kb":173901,"num_write":257,"num_write_kb":75129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":47}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":489,"reported_epoch":305,"state":"active+clean","last_fresh":"2023-06-27T12:43:18.508314+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:18.508314+0000","last_peered":"2023-06-27T12:43:18.508314+0000","last_clean":"2023-06-27T12:43:18.508314+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:18.508314+0000","last_undegraded":"2023-06-27T12:43:18.508314+0000","last_fullsized":"2023-06-27T12:43:18.508314+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"303'200","reported_seq":459,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.592469+0000","last_change":"2023-06-27T12:43:16.484809+0000","last_active":"2023-06-27T12:43:18.592469+0000","last_peered":"2023-06-27T12:43:18.592469+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:18.592469+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227208,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":86,"num_read_kb":47367,"num_write":63,"num_write_kb":17852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":251,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-06-27T12:43:18.505838+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:18.505838+0000","last_peered":"2023-06-27T12:43:18.505838+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:18.505838+0000","last_undegraded":"2023-06-27T12:43:18.505838+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"286'173","reported_seq":907,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.503373+0000","last_change":"2023-06-27T12:43:16.461032+0000","last_active":"2023-06-27T12:43:18.503373+0000","last_peered":"2023-06-27T12:43:18.503373+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:18.503373+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6134753,"num_objects":23,"num_object_clones":19,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":96,"num_read_kb":109556,"num_write":114,"num_write_kb":33633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"297'109","reported_seq":443,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.581337+0000","last_change":"2023-06-27T12:43:16.473212+0000","last_active":"2023-06-27T12:43:18.581337+0000","last_peered":"2023-06-27T12:43:18.581337+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:18.581337+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5815420,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":53576,"num_write":39,"num_write_kb":11125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":508,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-06-27T12:43:18.508859+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:18.508859+0000","last_peered":"2023-06-27T12:43:18.508859+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:18.508859+0000","last_undegraded":"2023-06-27T12:43:18.508859+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":473,"reported_epoch":305,"state":"active+clean","last_fresh":"2023-06-27T12:43:18.578949+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:18.578949+0000","last_peered":"2023-06-27T12:43:18.578949+0000","last_clean":"2023-06-27T12:43:18.578949+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:18.578949+0000","last_undegraded":"2023-06-27T12:43:18.578949+0000","last_fullsized":"2023-06-27T12:43:18.578949+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"305'220","reported_seq":924,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:18.643745+0000","last_change":"2023-06-27T12:43:16.459248+0000","last_active":"2023-06-27T12:43:18.643745+0000","last_peered":"2023-06-27T12:43:18.643745+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:18.643745+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832260,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":112358,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":101164534,"num_objects":310,"num_object_clones":263,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1841,"num_read_kb":1348534,"num_write":1645,"num_write_kb":480552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2930,"ondisk_log_size":2930,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":246068,"kb_used_data":211052,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94125772,"statfs":{"total":96636764160,"available":96384790528,"internally_reserved":0,"allocated":216117248,"data_stored":246989944,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":246824,"kb_used_data":211808,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94125016,"statfs":{"total":96636764160,"available":96384016384,"internally_reserved":0,"allocated":216891392,"data_stored":245474319,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":273788,"kb_used_data":238772,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94098052,"statfs":{"total":96636764160,"available":96356405248,"internally_reserved":0,"allocated":244502528,"data_stored":278567258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:22.512 INFO:tasks.rados.rados.0.smithi138.stdout:1257: done (12 left) 2023-06-27T12:43:22.513 INFO:tasks.rados.rados.0.smithi138.stdout:1258: read oid 17 snap -1 2023-06-27T12:43:22.513 INFO:tasks.rados.rados.0.smithi138.stdout:1258: expect (ObjNum 580 snap 139 seq_num 580) 2023-06-27T12:43:22.513 INFO:tasks.rados.rados.0.smithi138.stdout:1259: read oid 39 snap -1 2023-06-27T12:43:22.513 INFO:tasks.rados.rados.0.smithi138.stdout:1259: expect (ObjNum 595 snap 142 seq_num 595) 2023-06-27T12:43:22.513 INFO:tasks.rados.rados.0.smithi138.stdout:1260: write oid 10 current snap is 144 2023-06-27T12:43:22.514 INFO:tasks.rados.rados.0.smithi138.stdout:1260: seq_num 604 ranges {551428=642283,1694771=456657} 2023-06-27T12:43:22.519 INFO:tasks.rados.rados.0.smithi138.stdout:1260: writing smithi138138488-10 from 551428 to 1193711 tid 1 2023-06-27T12:43:22.525 INFO:tasks.rados.rados.0.smithi138.stdout:1260: writing smithi138138488-10 from 1694771 to 2151428 tid 2 2023-06-27T12:43:22.525 INFO:tasks.rados.rados.0.smithi138.stdout:1261: read oid 30 snap 138 2023-06-27T12:43:22.525 INFO:tasks.rados.rados.0.smithi138.stdout:1261: expect deleted 2023-06-27T12:43:22.525 INFO:tasks.rados.rados.0.smithi138.stdout: waiting on 16 2023-06-27T12:43:22.526 INFO:tasks.rados.rados.0.smithi138.stdout:1242: expect (ObjNum 596 snap 143 seq_num 596) 2023-06-27T12:43:22.674 INFO:tasks.rados.rados.0.smithi138.stdout:1242: done (15 left) 2023-06-27T12:43:22.674 INFO:tasks.rados.rados.0.smithi138.stdout:1243: done (14 left) 2023-06-27T12:43:22.674 INFO:tasks.rados.rados.0.smithi138.stdout:1262: read oid 41 snap -1 2023-06-27T12:43:22.674 INFO:tasks.rados.rados.0.smithi138.stdout:1262: expect deleted 2023-06-27T12:43:22.674 INFO:tasks.rados.rados.0.smithi138.stdout:1263: delete oid 23 current snap is 144 2023-06-27T12:43:22.674 INFO:tasks.rados.rados.0.smithi138.stdout:1248: finishing write tid 1 to smithi138138488-48 2023-06-27T12:43:22.675 INFO:tasks.rados.rados.0.smithi138.stdout:1247: finishing write tid 1 to smithi138138488-40 2023-06-27T12:43:22.675 INFO:tasks.rados.rados.0.smithi138.stdout:1245: finishing write tid 1 to smithi138138488-28 2023-06-27T12:43:22.675 INFO:tasks.rados.rados.0.smithi138.stdout:1247: finishing write tid 2 to smithi138138488-40 2023-06-27T12:43:22.675 INFO:tasks.rados.rados.0.smithi138.stdout:1247: finishing write tid 3 to smithi138138488-40 2023-06-27T12:43:22.675 INFO:tasks.rados.rados.0.smithi138.stdout:1245: finishing write tid 2 to smithi138138488-28 2023-06-27T12:43:22.675 INFO:tasks.rados.rados.0.smithi138.stdout:1247: finishing write tid 4 to smithi138138488-40 2023-06-27T12:43:22.675 INFO:tasks.rados.rados.0.smithi138.stdout:1247: oid 40 updating version 0 to 131 2023-06-27T12:43:22.675 INFO:tasks.rados.rados.0.smithi138.stdout:1247: oid 40 version 131 is already newer than 131 2023-06-27T12:43:22.676 INFO:tasks.rados.rados.0.smithi138.stdout:1247: oid 40 version 131 is already newer than 130 2023-06-27T12:43:22.676 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 131 (ObjNum 601 snap 144 seq_num 601) dirty exists 2023-06-27T12:43:22.676 INFO:tasks.rados.rados.0.smithi138.stdout:1247: left oid 40 (ObjNum 601 snap 144 seq_num 601) 2023-06-27T12:43:22.676 INFO:tasks.rados.rados.0.smithi138.stdout:1245: finishing write tid 3 to smithi138138488-28 2023-06-27T12:43:22.676 INFO:tasks.rados.rados.0.smithi138.stdout:1245: finishing write tid 4 to smithi138138488-28 2023-06-27T12:43:22.676 INFO:tasks.rados.rados.0.smithi138.stdout:1245: oid 28 updating version 0 to 170 2023-06-27T12:43:22.676 INFO:tasks.rados.rados.0.smithi138.stdout:1245: oid 28 updating version 170 to 172 2023-06-27T12:43:22.677 INFO:tasks.rados.rados.0.smithi138.stdout:1245: oid 28 version 172 is already newer than 172 2023-06-27T12:43:22.677 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 172 (ObjNum 600 snap 144 seq_num 600) dirty exists 2023-06-27T12:43:22.677 INFO:tasks.rados.rados.0.smithi138.stdout:1245: left oid 28 (ObjNum 600 snap 144 seq_num 600) 2023-06-27T12:43:22.677 INFO:tasks.rados.rados.0.smithi138.stdout:1248: finishing write tid 2 to smithi138138488-48 2023-06-27T12:43:22.677 INFO:tasks.rados.rados.0.smithi138.stdout:1248: finishing write tid 3 to smithi138138488-48 2023-06-27T12:43:22.677 INFO:tasks.rados.rados.0.smithi138.stdout:1248: finishing write tid 4 to smithi138138488-48 2023-06-27T12:43:22.677 INFO:tasks.rados.rados.0.smithi138.stdout:1248: oid 48 updating version 0 to 217 2023-06-27T12:43:22.677 INFO:tasks.rados.rados.0.smithi138.stdout:1248: oid 48 updating version 217 to 220 2023-06-27T12:43:22.678 INFO:tasks.rados.rados.0.smithi138.stdout:1248: oid 48 version 220 is already newer than 220 2023-06-27T12:43:22.678 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 220 (ObjNum 602 snap 144 seq_num 602) dirty exists 2023-06-27T12:43:22.678 INFO:tasks.rados.rados.0.smithi138.stdout:1248: left oid 48 (ObjNum 602 snap 144 seq_num 602) 2023-06-27T12:43:22.678 INFO:tasks.rados.rados.0.smithi138.stdout:1253: expect (ObjNum 562 snap 132 seq_num 562) 2023-06-27T12:43:22.882 INFO:tasks.rados.rados.0.smithi138.stdout:1251: finishing write tid 1 to smithi138138488-27 2023-06-27T12:43:22.882 INFO:tasks.rados.rados.0.smithi138.stdout:1251: finishing write tid 2 to smithi138138488-27 2023-06-27T12:43:22.882 INFO:tasks.rados.rados.0.smithi138.stdout:1251: finishing write tid 3 to smithi138138488-27 2023-06-27T12:43:22.882 INFO:tasks.rados.rados.0.smithi138.stdout:1251: finishing write tid 4 to smithi138138488-27 2023-06-27T12:43:22.883 INFO:tasks.rados.rados.0.smithi138.stdout:1251: oid 27 updating version 0 to 184 2023-06-27T12:43:22.883 INFO:tasks.rados.rados.0.smithi138.stdout:1251: oid 27 version 184 is already newer than 184 2023-06-27T12:43:22.883 INFO:tasks.rados.rados.0.smithi138.stdout:1251: oid 27 version 184 is already newer than 184 2023-06-27T12:43:22.883 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 184 (ObjNum 603 snap 144 seq_num 603) dirty exists 2023-06-27T12:43:22.883 INFO:tasks.rados.rados.0.smithi138.stdout:1251: left oid 27 (ObjNum 603 snap 144 seq_num 603) 2023-06-27T12:43:22.883 INFO:tasks.rados.rados.0.smithi138.stdout:1244: done (15 left) 2023-06-27T12:43:22.883 INFO:tasks.rados.rados.0.smithi138.stdout:1245: done (14 left) 2023-06-27T12:43:22.884 INFO:tasks.rados.rados.0.smithi138.stdout:1246: done (13 left) 2023-06-27T12:43:22.884 INFO:tasks.rados.rados.0.smithi138.stdout:1247: done (12 left) 2023-06-27T12:43:22.884 INFO:tasks.rados.rados.0.smithi138.stdout:1248: done (11 left) 2023-06-27T12:43:22.884 INFO:tasks.rados.rados.0.smithi138.stdout:1250: done (10 left) 2023-06-27T12:43:22.884 INFO:tasks.rados.rados.0.smithi138.stdout:1251: done (9 left) 2023-06-27T12:43:22.884 INFO:tasks.rados.rados.0.smithi138.stdout:1252: done (8 left) 2023-06-27T12:43:22.884 INFO:tasks.rados.rados.0.smithi138.stdout:1253: done (7 left) 2023-06-27T12:43:22.884 INFO:tasks.rados.rados.0.smithi138.stdout:1263: done (6 left) 2023-06-27T12:43:22.885 INFO:tasks.rados.rados.0.smithi138.stdout:1264: read oid 36 snap -1 2023-06-27T12:43:22.885 INFO:tasks.rados.rados.0.smithi138.stdout:1264: expect deleted 2023-06-27T12:43:22.885 INFO:tasks.rados.rados.0.smithi138.stdout:1265: snap_create 2023-06-27T12:43:22.885 INFO:tasks.rados.rados.0.smithi138.stdout:1255: expect (ObjNum 524 snap 121 seq_num 524) 2023-06-27T12:43:23.020 INFO:tasks.rados.rados.0.smithi138.stdout:1258: expect (ObjNum 580 snap 139 seq_num 580) 2023-06-27T12:43:23.157 INFO:tasks.rados.rados.0.smithi138.stdout:1260: finishing write tid 1 to smithi138138488-10 2023-06-27T12:43:23.158 INFO:tasks.rados.rados.0.smithi138.stdout:1259: expect (ObjNum 595 snap 142 seq_num 595) 2023-06-27T12:43:23.247 INFO:tasks.rados.rados.0.smithi138.stdout:1260: finishing write tid 2 to smithi138138488-10 2023-06-27T12:43:23.248 INFO:tasks.rados.rados.0.smithi138.stdout:1260: finishing write tid 3 to smithi138138488-10 2023-06-27T12:43:23.248 INFO:tasks.rados.rados.0.smithi138.stdout:1260: finishing write tid 4 to smithi138138488-10 2023-06-27T12:43:23.248 INFO:tasks.rados.rados.0.smithi138.stdout:1260: oid 10 updating version 0 to 178 2023-06-27T12:43:23.248 INFO:tasks.rados.rados.0.smithi138.stdout:1260: oid 10 version 178 is already newer than 175 2023-06-27T12:43:23.248 INFO:tasks.rados.rados.0.smithi138.stdout:1260: oid 10 version 178 is already newer than 178 2023-06-27T12:43:23.248 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 178 (ObjNum 604 snap 144 seq_num 604) dirty exists 2023-06-27T12:43:23.249 INFO:tasks.rados.rados.0.smithi138.stdout:1260: left oid 10 (ObjNum 604 snap 144 seq_num 604) 2023-06-27T12:43:23.518 INFO:tasks.rados.rados.0.smithi138.stdout:1255: done (7 left) 2023-06-27T12:43:23.519 INFO:tasks.rados.rados.0.smithi138.stdout:1258: done (6 left) 2023-06-27T12:43:23.519 INFO:tasks.rados.rados.0.smithi138.stdout:1259: done (5 left) 2023-06-27T12:43:23.519 INFO:tasks.rados.rados.0.smithi138.stdout:1260: done (4 left) 2023-06-27T12:43:23.519 INFO:tasks.rados.rados.0.smithi138.stdout:1261: done (3 left) 2023-06-27T12:43:23.519 INFO:tasks.rados.rados.0.smithi138.stdout:1262: done (2 left) 2023-06-27T12:43:23.519 INFO:tasks.rados.rados.0.smithi138.stdout:1264: done (1 left) 2023-06-27T12:43:23.519 INFO:tasks.rados.rados.0.smithi138.stdout:1265: done (0 left) 2023-06-27T12:43:23.519 INFO:tasks.rados.rados.0.smithi138.stdout:1266: write (excl) oid 40 current snap is 145 2023-06-27T12:43:23.520 INFO:tasks.rados.rados.0.smithi138.stdout:1266: seq_num 606 ranges {585042=792781,2095382=89660} 2023-06-27T12:43:23.525 INFO:tasks.rados.rados.0.smithi138.stdout:1266: writing smithi138138488-40 from 585042 to 1377823 tid 1 2023-06-27T12:43:23.530 INFO:tasks.rados.rados.0.smithi138.stdout:1266: writing smithi138138488-40 from 2095382 to 2185042 tid 2 2023-06-27T12:43:23.530 INFO:tasks.rados.rados.0.smithi138.stdout:1267: write (excl) oid 4 current snap is 145 2023-06-27T12:43:23.530 INFO:tasks.rados.rados.0.smithi138.stdout:1267: seq_num 607 ranges {601849=468030,1753864=447985} 2023-06-27T12:43:23.536 INFO:tasks.rados.rados.0.smithi138.stdout:1267: writing smithi138138488-4 from 601849 to 1069879 tid 1 2023-06-27T12:43:23.542 INFO:tasks.rados.rados.0.smithi138.stdout:1267: writing smithi138138488-4 from 1753864 to 2201849 tid 2 2023-06-27T12:43:23.543 INFO:tasks.rados.rados.0.smithi138.stdout:1268: write oid 30 current snap is 145 2023-06-27T12:43:23.543 INFO:tasks.rados.rados.0.smithi138.stdout:1268: seq_num 608 ranges {618656=543279,1812346=406310} 2023-06-27T12:43:23.549 INFO:tasks.rados.rados.0.smithi138.stdout:1268: writing smithi138138488-30 from 618656 to 1161935 tid 1 2023-06-27T12:43:23.555 INFO:tasks.rados.rados.0.smithi138.stdout:1268: writing smithi138138488-30 from 1812346 to 2218656 tid 2 2023-06-27T12:43:23.555 INFO:tasks.rados.rados.0.smithi138.stdout:1269: read oid 44 snap -1 2023-06-27T12:43:23.555 INFO:tasks.rados.rados.0.smithi138.stdout:1269: expect deleted 2023-06-27T12:43:23.556 INFO:tasks.rados.rados.0.smithi138.stdout:1270: write oid 41 current snap is 145 2023-06-27T12:43:23.556 INFO:tasks.rados.rados.0.smithi138.stdout:1270: seq_num 609 ranges {635463=734881,1987181=248282} 2023-06-27T12:43:23.563 INFO:tasks.rados.rados.0.smithi138.stdout:1270: writing smithi138138488-41 from 635463 to 1370344 tid 1 2023-06-27T12:43:23.568 INFO:tasks.rados.rados.0.smithi138.stdout:1270: writing smithi138138488-41 from 1987181 to 2235463 tid 2 2023-06-27T12:43:23.568 INFO:tasks.rados.rados.0.smithi138.stdout:1271: write oid 29 current snap is 145 2023-06-27T12:43:23.568 INFO:tasks.rados.rados.0.smithi138.stdout:1271: seq_num 610 ranges {652270=410130,1529310=493867} 2023-06-27T12:43:23.573 INFO:tasks.rados.rados.0.smithi138.stdout:1271: writing smithi138138488-29 from 652270 to 1062400 tid 1 2023-06-27T12:43:23.579 INFO:tasks.rados.rados.0.smithi138.stdout:1271: writing smithi138138488-29 from 1529310 to 2023177 tid 2 2023-06-27T12:43:23.579 INFO:tasks.rados.rados.0.smithi138.stdout:1272: write (excl) oid 1 current snap is 145 2023-06-27T12:43:23.579 INFO:tasks.rados.rados.0.smithi138.stdout:1272: seq_num 611 ranges {669077=485379,1587792=681285} 2023-06-27T12:43:23.585 INFO:tasks.rados.rados.0.smithi138.stdout:1272: writing smithi138138488-1 from 669077 to 1154456 tid 1 2023-06-27T12:43:23.592 INFO:tasks.rados.rados.0.smithi138.stdout:1272: writing smithi138138488-1 from 1587792 to 2269077 tid 2 2023-06-27T12:43:23.592 INFO:tasks.rados.rados.0.smithi138.stdout:1273: snap_remove snap 144 2023-06-27T12:43:24.140 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:24.268 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:24.263+0000 7f5f2adb5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:24.271 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:24.267+0000 7f5f2adb5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:24.321 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:24.324 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:24.325 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:24.524 INFO:tasks.rados.rados.0.smithi138.stdout:1273: done (7 left) 2023-06-27T12:43:24.524 INFO:tasks.rados.rados.0.smithi138.stdout:1274: write oid 26 current snap is 145 2023-06-27T12:43:24.524 INFO:tasks.rados.rados.0.smithi138.stdout:1274: seq_num 612 ranges {685884=560628,2046274=239610} 2023-06-27T12:43:24.530 INFO:tasks.rados.rados.0.smithi138.stdout:1274: writing smithi138138488-26 from 685884 to 1246512 tid 1 2023-06-27T12:43:24.537 INFO:tasks.rados.rados.0.smithi138.stdout:1274: writing smithi138138488-26 from 2046274 to 2285884 tid 2 2023-06-27T12:43:24.537 INFO:tasks.rados.rados.0.smithi138.stdout:1275: write (excl) oid 31 current snap is 145 2023-06-27T12:43:24.537 INFO:tasks.rados.rados.0.smithi138.stdout:1267: finishing write tid 1 to smithi138138488-4 2023-06-27T12:43:24.537 INFO:tasks.rados.rados.0.smithi138.stdout:1270: finishing write tid 1 to smithi138138488-41 2023-06-27T12:43:24.537 INFO:tasks.rados.rados.0.smithi138.stdout:1270: finishing write tid 2 to smithi138138488-41 2023-06-27T12:43:24.537 INFO:tasks.rados.rados.0.smithi138.stdout:1275: seq_num 613 ranges {702691=635877,2104756=197935} 2023-06-27T12:43:24.544 INFO:tasks.rados.rados.0.smithi138.stdout:1275: writing smithi138138488-31 from 702691 to 1338568 tid 1 2023-06-27T12:43:24.550 INFO:tasks.rados.rados.0.smithi138.stdout:1275: writing smithi138138488-31 from 2104756 to 2302691 tid 2 2023-06-27T12:43:24.550 INFO:tasks.rados.rados.0.smithi138.stdout:1269: done (8 left) 2023-06-27T12:43:24.550 INFO:tasks.rados.rados.0.smithi138.stdout:1276: read oid 14 snap -1 2023-06-27T12:43:24.550 INFO:tasks.rados.rados.0.smithi138.stdout:1276: expect deleted 2023-06-27T12:43:24.551 INFO:tasks.rados.rados.0.smithi138.stdout:1277: write oid 21 current snap is 145 2023-06-27T12:43:24.551 INFO:tasks.rados.rados.0.smithi138.stdout:1266: finishing write tid 1 to smithi138138488-40 2023-06-27T12:43:24.551 INFO:tasks.rados.rados.0.smithi138.stdout:1277: seq_num 614 ranges {719498=711126,2046885=272613} 2023-06-27T12:43:24.558 INFO:tasks.rados.rados.0.smithi138.stdout:1277: writing smithi138138488-21 from 719498 to 1430624 tid 1 2023-06-27T12:43:24.563 INFO:tasks.rados.rados.0.smithi138.stdout:1277: writing smithi138138488-21 from 2046885 to 2319498 tid 2 2023-06-27T12:43:24.563 INFO:tasks.rados.rados.0.smithi138.stdout:1278: snap_remove snap 140 2023-06-27T12:43:25.448 INFO:tasks.rados.rados.0.smithi138.stdout:1278: done (10 left) 2023-06-27T12:43:25.448 INFO:tasks.rados.rados.0.smithi138.stdout:1279: delete oid 18 current snap is 145 2023-06-27T12:43:25.448 INFO:tasks.rados.rados.0.smithi138.stdout:1270: finishing write tid 3 to smithi138138488-41 2023-06-27T12:43:25.449 INFO:tasks.rados.rados.0.smithi138.stdout:1270: finishing write tid 4 to smithi138138488-41 2023-06-27T12:43:25.449 INFO:tasks.rados.rados.0.smithi138.stdout:1270: oid 41 updating version 0 to 179 2023-06-27T12:43:25.449 INFO:tasks.rados.rados.0.smithi138.stdout:1270: oid 41 version 179 is already newer than 179 2023-06-27T12:43:25.449 INFO:tasks.rados.rados.0.smithi138.stdout:1270: oid 41 version 179 is already newer than 179 2023-06-27T12:43:25.449 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 41 v 179 (ObjNum 609 snap 145 seq_num 609) dirty exists 2023-06-27T12:43:25.449 INFO:tasks.rados.rados.0.smithi138.stdout:1270: left oid 41 (ObjNum 609 snap 145 seq_num 609) 2023-06-27T12:43:25.449 INFO:tasks.rados.rados.0.smithi138.stdout:1271: finishing write tid 1 to smithi138138488-29 2023-06-27T12:43:25.450 INFO:tasks.rados.rados.0.smithi138.stdout:1266: finishing write tid 2 to smithi138138488-40 2023-06-27T12:43:25.450 INFO:tasks.rados.rados.0.smithi138.stdout:1266: finishing write tid 3 to smithi138138488-40 2023-06-27T12:43:25.450 INFO:tasks.rados.rados.0.smithi138.stdout:1266: finishing write tid 4 to smithi138138488-40 2023-06-27T12:43:25.450 INFO:tasks.rados.rados.0.smithi138.stdout:1266: oid 40 updating version 0 to 141 2023-06-27T12:43:25.450 INFO:tasks.rados.rados.0.smithi138.stdout:1266: oid 40 version 141 is already newer than 138 2023-06-27T12:43:25.450 INFO:tasks.rados.rados.0.smithi138.stdout:1266: oid 40 version 141 is already newer than 141 2023-06-27T12:43:25.450 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 141 (ObjNum 606 snap 145 seq_num 606) dirty exists 2023-06-27T12:43:25.450 INFO:tasks.rados.rados.0.smithi138.stdout:1266: left oid 40 (ObjNum 606 snap 145 seq_num 606) 2023-06-27T12:43:25.451 INFO:tasks.rados.rados.0.smithi138.stdout:1271: finishing write tid 2 to smithi138138488-29 2023-06-27T12:43:25.451 INFO:tasks.rados.rados.0.smithi138.stdout:1272: finishing write tid 1 to smithi138138488-1 2023-06-27T12:43:25.451 INFO:tasks.rados.rados.0.smithi138.stdout:1271: finishing write tid 3 to smithi138138488-29 2023-06-27T12:43:25.451 INFO:tasks.rados.rados.0.smithi138.stdout:1267: finishing write tid 2 to smithi138138488-4 2023-06-27T12:43:25.451 INFO:tasks.rados.rados.0.smithi138.stdout:1267: finishing write tid 3 to smithi138138488-4 2023-06-27T12:43:25.451 INFO:tasks.rados.rados.0.smithi138.stdout:1271: finishing write tid 4 to smithi138138488-29 2023-06-27T12:43:25.451 INFO:tasks.rados.rados.0.smithi138.stdout:1271: oid 29 updating version 0 to 115 2023-06-27T12:43:25.451 INFO:tasks.rados.rados.0.smithi138.stdout:1271: oid 29 version 115 is already newer than 115 2023-06-27T12:43:25.452 INFO:tasks.rados.rados.0.smithi138.stdout:1271: oid 29 version 115 is already newer than 115 2023-06-27T12:43:25.452 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 115 (ObjNum 610 snap 145 seq_num 610) dirty exists 2023-06-27T12:43:25.452 INFO:tasks.rados.rados.0.smithi138.stdout:1271: left oid 29 (ObjNum 610 snap 145 seq_num 610) 2023-06-27T12:43:25.452 INFO:tasks.rados.rados.0.smithi138.stdout:1267: finishing write tid 4 to smithi138138488-4 2023-06-27T12:43:25.452 INFO:tasks.rados.rados.0.smithi138.stdout:1267: oid 4 updating version 0 to 204 2023-06-27T12:43:25.452 INFO:tasks.rados.rados.0.smithi138.stdout:1267: oid 4 version 204 is already newer than 204 2023-06-27T12:43:25.452 INFO:tasks.rados.rados.0.smithi138.stdout:1267: oid 4 version 204 is already newer than 201 2023-06-27T12:43:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 204 (ObjNum 607 snap 145 seq_num 607) dirty exists 2023-06-27T12:43:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:1267: left oid 4 (ObjNum 607 snap 145 seq_num 607) 2023-06-27T12:43:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:1272: finishing write tid 2 to smithi138138488-1 2023-06-27T12:43:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:1272: finishing write tid 3 to smithi138138488-1 2023-06-27T12:43:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:1268: finishing write tid 1 to smithi138138488-30 2023-06-27T12:43:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:1272: finishing write tid 4 to smithi138138488-1 2023-06-27T12:43:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:1272: oid 1 updating version 0 to 361 2023-06-27T12:43:25.453 INFO:tasks.rados.rados.0.smithi138.stdout:1272: oid 1 version 361 is already newer than 361 2023-06-27T12:43:25.454 INFO:tasks.rados.rados.0.smithi138.stdout:1272: oid 1 version 361 is already newer than 359 2023-06-27T12:43:25.454 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 361 (ObjNum 611 snap 145 seq_num 611) dirty exists 2023-06-27T12:43:25.454 INFO:tasks.rados.rados.0.smithi138.stdout:1272: left oid 1 (ObjNum 611 snap 145 seq_num 611) 2023-06-27T12:43:25.454 INFO:tasks.rados.rados.0.smithi138.stdout:1268: finishing write tid 2 to smithi138138488-30 2023-06-27T12:43:25.454 INFO:tasks.rados.rados.0.smithi138.stdout:1268: finishing write tid 3 to smithi138138488-30 2023-06-27T12:43:25.454 INFO:tasks.rados.rados.0.smithi138.stdout:1268: finishing write tid 4 to smithi138138488-30 2023-06-27T12:43:25.454 INFO:tasks.rados.rados.0.smithi138.stdout:1268: oid 30 updating version 0 to 207 2023-06-27T12:43:25.454 INFO:tasks.rados.rados.0.smithi138.stdout:1268: oid 30 version 207 is already newer than 207 2023-06-27T12:43:25.455 INFO:tasks.rados.rados.0.smithi138.stdout:1268: oid 30 version 207 is already newer than 207 2023-06-27T12:43:25.455 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 207 (ObjNum 608 snap 145 seq_num 608) dirty exists 2023-06-27T12:43:25.455 INFO:tasks.rados.rados.0.smithi138.stdout:1268: left oid 30 (ObjNum 608 snap 145 seq_num 608) 2023-06-27T12:43:25.455 INFO:tasks.rados.rados.0.smithi138.stdout:1274: finishing write tid 1 to smithi138138488-26 2023-06-27T12:43:25.455 INFO:tasks.rados.rados.0.smithi138.stdout:1274: finishing write tid 2 to smithi138138488-26 2023-06-27T12:43:25.455 INFO:tasks.rados.rados.0.smithi138.stdout:1274: finishing write tid 3 to smithi138138488-26 2023-06-27T12:43:25.455 INFO:tasks.rados.rados.0.smithi138.stdout:1274: finishing write tid 4 to smithi138138488-26 2023-06-27T12:43:25.455 INFO:tasks.rados.rados.0.smithi138.stdout:1274: oid 26 updating version 0 to 182 2023-06-27T12:43:25.456 INFO:tasks.rados.rados.0.smithi138.stdout:1274: oid 26 version 182 is already newer than 182 2023-06-27T12:43:25.456 INFO:tasks.rados.rados.0.smithi138.stdout:1274: oid 26 version 182 is already newer than 182 2023-06-27T12:43:25.456 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 182 (ObjNum 612 snap 145 seq_num 612) dirty exists 2023-06-27T12:43:25.456 INFO:tasks.rados.rados.0.smithi138.stdout:1274: left oid 26 (ObjNum 612 snap 145 seq_num 612) 2023-06-27T12:43:25.456 INFO:tasks.rados.rados.0.smithi138.stdout:1277: finishing write tid 1 to smithi138138488-21 2023-06-27T12:43:25.456 INFO:tasks.rados.rados.0.smithi138.stdout:1275: finishing write tid 1 to smithi138138488-31 2023-06-27T12:43:25.456 INFO:tasks.rados.rados.0.smithi138.stdout:1277: finishing write tid 2 to smithi138138488-21 2023-06-27T12:43:25.457 INFO:tasks.rados.rados.0.smithi138.stdout:1277: finishing write tid 3 to smithi138138488-21 2023-06-27T12:43:25.457 INFO:tasks.rados.rados.0.smithi138.stdout:1277: finishing write tid 4 to smithi138138488-21 2023-06-27T12:43:25.457 INFO:tasks.rados.rados.0.smithi138.stdout:1277: oid 21 updating version 0 to 364 2023-06-27T12:43:25.457 INFO:tasks.rados.rados.0.smithi138.stdout:1277: oid 21 version 364 is already newer than 364 2023-06-27T12:43:25.457 INFO:tasks.rados.rados.0.smithi138.stdout:1277: oid 21 version 364 is already newer than 364 2023-06-27T12:43:25.457 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 21 v 364 (ObjNum 614 snap 145 seq_num 614) dirty exists 2023-06-27T12:43:25.457 INFO:tasks.rados.rados.0.smithi138.stdout:1277: left oid 21 (ObjNum 614 snap 145 seq_num 614) 2023-06-27T12:43:25.458 INFO:tasks.rados.rados.0.smithi138.stdout:1275: finishing write tid 2 to smithi138138488-31 2023-06-27T12:43:25.458 INFO:tasks.rados.rados.0.smithi138.stdout:1275: finishing write tid 3 to smithi138138488-31 2023-06-27T12:43:25.458 INFO:tasks.rados.rados.0.smithi138.stdout:1275: finishing write tid 4 to smithi138138488-31 2023-06-27T12:43:25.458 INFO:tasks.rados.rados.0.smithi138.stdout:1275: oid 31 updating version 0 to 378 2023-06-27T12:43:25.458 INFO:tasks.rados.rados.0.smithi138.stdout:1275: oid 31 version 378 is already newer than 378 2023-06-27T12:43:25.458 INFO:tasks.rados.rados.0.smithi138.stdout:1275: oid 31 version 378 is already newer than 375 2023-06-27T12:43:25.458 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 378 (ObjNum 613 snap 145 seq_num 613) dirty exists 2023-06-27T12:43:25.459 INFO:tasks.rados.rados.0.smithi138.stdout:1275: left oid 31 (ObjNum 613 snap 145 seq_num 613) 2023-06-27T12:43:25.493 INFO:tasks.rados.rados.0.smithi138.stdout:1266: done (10 left) 2023-06-27T12:43:25.493 INFO:tasks.rados.rados.0.smithi138.stdout:1267: done (9 left) 2023-06-27T12:43:25.494 INFO:tasks.rados.rados.0.smithi138.stdout:1268: done (8 left) 2023-06-27T12:43:25.494 INFO:tasks.rados.rados.0.smithi138.stdout:1270: done (7 left) 2023-06-27T12:43:25.494 INFO:tasks.rados.rados.0.smithi138.stdout:1271: done (6 left) 2023-06-27T12:43:25.494 INFO:tasks.rados.rados.0.smithi138.stdout:1272: done (5 left) 2023-06-27T12:43:25.494 INFO:tasks.rados.rados.0.smithi138.stdout:1274: done (4 left) 2023-06-27T12:43:25.494 INFO:tasks.rados.rados.0.smithi138.stdout:1275: done (3 left) 2023-06-27T12:43:25.494 INFO:tasks.rados.rados.0.smithi138.stdout:1276: done (2 left) 2023-06-27T12:43:25.495 INFO:tasks.rados.rados.0.smithi138.stdout:1277: done (1 left) 2023-06-27T12:43:25.495 INFO:tasks.rados.rados.0.smithi138.stdout:1279: done (0 left) 2023-06-27T12:43:25.495 INFO:tasks.rados.rados.0.smithi138.stdout:1280: snap_create 2023-06-27T12:43:25.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~171s 2023-06-27T12:43:26.326 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:26.450 INFO:tasks.rados.rados.0.smithi138.stdout:1280: done (0 left) 2023-06-27T12:43:26.451 INFO:tasks.rados.rados.0.smithi138.stdout:1281: snap_remove snap 138 2023-06-27T12:43:26.473 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:26.468+0000 7f230a791700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:26.476 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:26.472+0000 7f230a791700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:26.524 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:26.525 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:27.453 INFO:tasks.rados.rados.0.smithi138.stdout:1281: done (0 left) 2023-06-27T12:43:27.453 INFO:tasks.rados.rados.0.smithi138.stdout:1282: write (excl) oid 43 current snap is 146 2023-06-27T12:43:27.453 INFO:tasks.rados.rados.0.smithi138.stdout:1282: seq_num 616 ranges {753112=577977,1880202=472910} 2023-06-27T12:43:27.460 INFO:tasks.rados.rados.0.smithi138.stdout:1282: writing smithi138138488-43 from 753112 to 1331089 tid 1 2023-06-27T12:43:27.466 INFO:tasks.rados.rados.0.smithi138.stdout:1282: writing smithi138138488-43 from 1880202 to 2353112 tid 2 2023-06-27T12:43:27.466 INFO:tasks.rados.rados.0.smithi138.stdout:1283: snap_remove snap 145 2023-06-27T12:43:27.474 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:27.474 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:27.474 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:27.626 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:27.618+0000 7f6e19c94700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:27.629 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:27.625+0000 7f6e19c94700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:27.845 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:27.847 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:27.864 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2023-06-27T12:43:27.447278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":100391907,"num_objects":316,"num_object_clones":269,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1870,"num_read_kb":1373587,"num_write":1677,"num_write_kb":488340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3077,"ondisk_log_size":3077,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":962956,"kb_used_data":826988,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376524404,"statfs":{"total":386547056640,"available":385560989696,"internally_reserved":0,"allocated":846835712,"data_stored":964064545,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-772627,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":29,"num_read_kb":25053,"num_write":32,"num_write_kb":7788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.933251"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":742,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.532582+0000","last_change":"2023-06-27T12:43:22.541584+0000","last_active":"2023-06-27T12:43:23.532582+0000","last_peered":"2023-06-27T12:43:23.532582+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:23.532582+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"306'184","reported_seq":976,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.531965+0000","last_change":"2023-06-27T12:43:22.545583+0000","last_active":"2023-06-27T12:43:23.531965+0000","last_peered":"2023-06-27T12:43:23.531965+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:23.531965+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13042628,"num_objects":23,"num_object_clones":19,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":170366,"num_write":133,"num_write_kb":39302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"301'254","reported_seq":967,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.564129+0000","last_change":"2023-06-27T12:43:23.564129+0000","last_active":"2023-06-27T12:43:23.564129+0000","last_peered":"2023-06-27T12:43:23.564129+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:23.564129+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5462473,"num_objects":30,"num_object_clones":25,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":206,"num_read_kb":131103,"num_write":165,"num_write_kb":47622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"310'141","reported_seq":911,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.666383+0000","last_change":"2023-06-27T12:43:22.587306+0000","last_active":"2023-06-27T12:43:23.666383+0000","last_peered":"2023-06-27T12:43:23.666383+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:23.666383+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8924847,"num_objects":23,"num_object_clones":19,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":96601,"num_write":106,"num_write_kb":29985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":23}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"305'91","reported_seq":466,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.553178+0000","last_change":"2023-06-27T12:43:22.545412+0000","last_active":"2023-06-27T12:43:23.553178+0000","last_peered":"2023-06-27T12:43:23.553178+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:23.553178+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3848902,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":88,"num_read_kb":50182,"num_write":49,"num_write_kb":15136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"299'371","reported_seq":996,"reported_epoch":310,"state":"active+clean","last_fresh":"2023-06-27T12:43:23.531865+0000","last_change":"2023-06-27T12:43:22.611965+0000","last_active":"2023-06-27T12:43:23.531865+0000","last_peered":"2023-06-27T12:43:23.531865+0000","last_clean":"2023-06-27T12:43:23.531865+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:23.531865+0000","last_undegraded":"2023-06-27T12:43:23.531865+0000","last_fullsized":"2023-06-27T12:43:23.531865+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10823972,"num_objects":41,"num_object_clones":35,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":298,"num_read_kb":193210,"num_write":220,"num_write_kb":63975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"309'178","reported_seq":810,"reported_epoch":310,"state":"active+clean","last_fresh":"2023-06-27T12:43:23.559312+0000","last_change":"2023-06-27T12:43:22.634989+0000","last_active":"2023-06-27T12:43:23.559312+0000","last_peered":"2023-06-27T12:43:23.559312+0000","last_clean":"2023-06-27T12:43:23.559312+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:23.559312+0000","last_undegraded":"2023-06-27T12:43:23.559312+0000","last_fullsized":"2023-06-27T12:43:23.559312+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8050916,"num_objects":21,"num_object_clones":18,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":78360,"num_write":108,"num_write_kb":31422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":777,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.549973+0000","last_change":"2023-06-27T12:43:22.542538+0000","last_active":"2023-06-27T12:43:23.549973+0000","last_peered":"2023-06-27T12:43:23.549973+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:23.549973+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":518,"reported_epoch":310,"state":"active+undersized","last_fresh":"2023-06-27T12:43:23.528877+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:23.528877+0000","last_peered":"2023-06-27T12:43:23.528877+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:23.528877+0000","last_undegraded":"2023-06-27T12:43:23.528877+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":691,"reported_epoch":310,"state":"active+undersized","last_fresh":"2023-06-27T12:43:23.543744+0000","last_change":"2023-06-27T12:43:22.574585+0000","last_active":"2023-06-27T12:43:23.543744+0000","last_peered":"2023-06-27T12:43:23.543744+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:23.543744+0000","last_undegraded":"2023-06-27T12:43:23.543744+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":499,"reported_epoch":310,"state":"active+clean","last_fresh":"2023-06-27T12:43:23.548710+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:23.548710+0000","last_peered":"2023-06-27T12:43:23.548710+0000","last_clean":"2023-06-27T12:43:23.548710+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:23.548710+0000","last_undegraded":"2023-06-27T12:43:23.548710+0000","last_fullsized":"2023-06-27T12:43:23.548710+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"309'171","reported_seq":893,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.530369+0000","last_change":"2023-06-27T12:43:22.553727+0000","last_active":"2023-06-27T12:43:23.530369+0000","last_peered":"2023-06-27T12:43:23.530369+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:23.530369+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806868,"num_objects":20,"num_object_clones":15,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":72,"num_read_kb":70434,"num_write":106,"num_write_kb":29767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":484,"reported_epoch":310,"state":"active+undersized","last_fresh":"2023-06-27T12:43:23.544770+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:23.544770+0000","last_peered":"2023-06-27T12:43:23.544770+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:23.544770+0000","last_undegraded":"2023-06-27T12:43:23.544770+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"304'404","reported_seq":563,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.526699+0000","last_change":"2023-06-27T12:43:22.550488+0000","last_active":"2023-06-27T12:43:23.526699+0000","last_peered":"2023-06-27T12:43:23.526699+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:23.526699+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9025656,"num_objects":21,"num_object_clones":19,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":56,"num_read_kb":33573,"num_write":108,"num_write_kb":33405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":481,"reported_epoch":310,"state":"active+clean","last_fresh":"2023-06-27T12:43:23.554200+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:23.554200+0000","last_peered":"2023-06-27T12:43:23.554200+0000","last_clean":"2023-06-27T12:43:23.554200+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:23.554200+0000","last_undegraded":"2023-06-27T12:43:23.554200+0000","last_fullsized":"2023-06-27T12:43:23.554200+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":251,"reported_epoch":310,"state":"active+undersized","last_fresh":"2023-06-27T12:43:23.542440+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:23.542440+0000","last_peered":"2023-06-27T12:43:23.542440+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:23.542440+0000","last_undegraded":"2023-06-27T12:43:23.542440+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"310'361","reported_seq":1099,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.715716+0000","last_change":"2023-06-27T12:43:23.557898+0000","last_active":"2023-06-27T12:43:23.715716+0000","last_peered":"2023-06-27T12:43:23.715716+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:23.715716+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6252435,"num_objects":47,"num_object_clones":41,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":289,"num_read_kb":173902,"num_write":261,"num_write_kb":76270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":47}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":499,"reported_epoch":310,"state":"active+clean","last_fresh":"2023-06-27T12:43:23.555398+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:23.555398+0000","last_peered":"2023-06-27T12:43:23.555398+0000","last_clean":"2023-06-27T12:43:23.555398+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:23.555398+0000","last_undegraded":"2023-06-27T12:43:23.555398+0000","last_fullsized":"2023-06-27T12:43:23.555398+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"310'207","reported_seq":483,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.723855+0000","last_change":"2023-06-27T12:43:22.589771+0000","last_active":"2023-06-27T12:43:23.723855+0000","last_peered":"2023-06-27T12:43:23.723855+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:23.723855+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6571966,"num_objects":13,"num_object_clones":13,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":53231,"num_write":71,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":13}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":261,"reported_epoch":310,"state":"active+undersized","last_fresh":"2023-06-27T12:43:23.526407+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:23.526407+0000","last_peered":"2023-06-27T12:43:23.526407+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:23.526407+0000","last_undegraded":"2023-06-27T12:43:23.526407+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"310'179","reported_seq":925,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.572873+0000","last_change":"2023-06-27T12:43:22.536890+0000","last_active":"2023-06-27T12:43:23.572873+0000","last_peered":"2023-06-27T12:43:23.572873+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:23.572873+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8370216,"num_objects":23,"num_object_clones":19,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":97,"num_read_kb":109557,"num_write":118,"num_write_kb":34594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"310'115","reported_seq":460,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.615238+0000","last_change":"2023-06-27T12:43:22.546813+0000","last_active":"2023-06-27T12:43:23.615238+0000","last_peered":"2023-06-27T12:43:23.615238+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:23.615238+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933102,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":53577,"num_write":43,"num_write_kb":12009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":518,"reported_epoch":310,"state":"active+undersized","last_fresh":"2023-06-27T12:43:23.528098+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:23.528098+0000","last_peered":"2023-06-27T12:43:23.528098+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:23.528098+0000","last_undegraded":"2023-06-27T12:43:23.528098+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":481,"reported_epoch":310,"state":"active+clean","last_fresh":"2023-06-27T12:43:23.549111+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:23.549111+0000","last_peered":"2023-06-27T12:43:23.549111+0000","last_clean":"2023-06-27T12:43:23.549111+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:23.549111+0000","last_undegraded":"2023-06-27T12:43:23.549111+0000","last_fullsized":"2023-06-27T12:43:23.549111+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"305'220","reported_seq":941,"reported_epoch":310,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:23.525313+0000","last_change":"2023-06-27T12:43:22.544079+0000","last_active":"2023-06-27T12:43:23.525313+0000","last_peered":"2023-06-27T12:43:23.525313+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:23.525313+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832260,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":112358,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":100391907,"num_objects":316,"num_object_clones":269,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1870,"num_read_kb":1373587,"num_write":1677,"num_write_kb":488340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2980,"ondisk_log_size":2980,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673290,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":250076,"kb_used_data":215060,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94121764,"statfs":{"total":96636764160,"available":96380686336,"internally_reserved":0,"allocated":220221440,"data_stored":251860475,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":252712,"kb_used_data":217696,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94119128,"statfs":{"total":96636764160,"available":96377987072,"internally_reserved":0,"allocated":222920704,"data_stored":251648204,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673290,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":278888,"kb_used_data":243872,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94092952,"statfs":{"total":96636764160,"available":96351182848,"internally_reserved":0,"allocated":249724928,"data_stored":283516314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:27.867 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.b 3.8 3.7 3.6 2023-06-27T12:43:27.999 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:27.987+0000 7f561cb34700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:28.002 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:27.999+0000 7f561cb34700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:28.197 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.f] on osd.0 to force-recovery; instructing pg(s) [3.7,3.8] on osd.1 to force-recovery; instructing pg(s) [3.6] on osd.2 to force-recovery; 2023-06-27T12:43:28.197 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:28.466 INFO:tasks.rados.rados.0.smithi138.stdout:1283: done (1 left) 2023-06-27T12:43:28.466 INFO:tasks.rados.rados.0.smithi138.stdout:1284: delete oid 28 current snap is 146 2023-06-27T12:43:28.467 INFO:tasks.rados.rados.0.smithi138.stdout:1282: finishing write tid 1 to smithi138138488-43 2023-06-27T12:43:28.467 INFO:tasks.rados.rados.0.smithi138.stdout:1282: finishing write tid 2 to smithi138138488-43 2023-06-27T12:43:28.467 INFO:tasks.rados.rados.0.smithi138.stdout:1282: finishing write tid 3 to smithi138138488-43 2023-06-27T12:43:28.468 INFO:tasks.rados.rados.0.smithi138.stdout:1282: finishing write tid 4 to smithi138138488-43 2023-06-27T12:43:28.468 INFO:tasks.rados.rados.0.smithi138.stdout:1282: oid 43 updating version 0 to 186 2023-06-27T12:43:28.468 INFO:tasks.rados.rados.0.smithi138.stdout:1282: oid 43 version 186 is already newer than 183 2023-06-27T12:43:28.468 INFO:tasks.rados.rados.0.smithi138.stdout:1282: oid 43 version 186 is already newer than 186 2023-06-27T12:43:28.469 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 186 (ObjNum 616 snap 146 seq_num 616) dirty exists 2023-06-27T12:43:28.469 INFO:tasks.rados.rados.0.smithi138.stdout:1282: left oid 43 (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:43:28.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:28.527 INFO:tasks.rados.rados.0.smithi138.stdout:1282: done (1 left) 2023-06-27T12:43:28.527 INFO:tasks.rados.rados.0.smithi138.stdout:1284: done (0 left) 2023-06-27T12:43:28.528 INFO:tasks.rados.rados.0.smithi138.stdout:1285: delete oid 45 current snap is 146 2023-06-27T12:43:28.601 INFO:tasks.rados.rados.0.smithi138.stdout:1285: done (0 left) 2023-06-27T12:43:28.601 INFO:tasks.rados.rados.0.smithi138.stdout:1286: read oid 27 snap -1 2023-06-27T12:43:28.601 INFO:tasks.rados.rados.0.smithi138.stdout:1286: expect (ObjNum 603 snap 144 seq_num 603) 2023-06-27T12:43:28.601 INFO:tasks.rados.rados.0.smithi138.stdout:1287: write (excl) oid 30 current snap is 146 2023-06-27T12:43:28.601 INFO:tasks.rados.rados.0.smithi138.stdout:1287: seq_num 619 ranges {403533=403724,1539295=652201} 2023-06-27T12:43:28.605 INFO:tasks.rados.rados.0.smithi138.stdout:1287: writing smithi138138488-30 from 403533 to 807257 tid 1 2023-06-27T12:43:28.615 INFO:tasks.rados.rados.0.smithi138.stdout:1287: writing smithi138138488-30 from 1539295 to 2191496 tid 2 2023-06-27T12:43:28.615 INFO:tasks.rados.rados.0.smithi138.stdout:1288: delete oid 18 current snap is 146 2023-06-27T12:43:28.628 INFO:tasks.rados.rados.0.smithi138.stdout:1286: expect (ObjNum 603 snap 144 seq_num 603) 2023-06-27T12:43:28.660 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:28.655+0000 7f4a6e27d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:28.663 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:28.659+0000 7f4a6e27d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:28.709 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:28.712 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:28.713 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:28.733 INFO:tasks.rados.rados.0.smithi138.stdout:1287: finishing write tid 1 to smithi138138488-30 2023-06-27T12:43:28.733 INFO:tasks.rados.rados.0.smithi138.stdout:1287: finishing write tid 2 to smithi138138488-30 2023-06-27T12:43:28.734 INFO:tasks.rados.rados.0.smithi138.stdout:1287: finishing write tid 3 to smithi138138488-30 2023-06-27T12:43:28.734 INFO:tasks.rados.rados.0.smithi138.stdout:1287: finishing write tid 4 to smithi138138488-30 2023-06-27T12:43:28.734 INFO:tasks.rados.rados.0.smithi138.stdout:1287: oid 30 updating version 0 to 212 2023-06-27T12:43:28.734 INFO:tasks.rados.rados.0.smithi138.stdout:1287: oid 30 version 212 is already newer than 210 2023-06-27T12:43:28.734 INFO:tasks.rados.rados.0.smithi138.stdout:1287: oid 30 version 212 is already newer than 212 2023-06-27T12:43:28.734 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 212 (ObjNum 619 snap 146 seq_num 619) dirty exists 2023-06-27T12:43:28.734 INFO:tasks.rados.rados.0.smithi138.stdout:1287: left oid 30 (ObjNum 619 snap 146 seq_num 619) 2023-06-27T12:43:28.735 INFO:tasks.rados.rados.0.smithi138.stdout:1286: done (2 left) 2023-06-27T12:43:28.735 INFO:tasks.rados.rados.0.smithi138.stdout:1287: done (1 left) 2023-06-27T12:43:28.735 INFO:tasks.rados.rados.0.smithi138.stdout:1288: done (0 left) 2023-06-27T12:43:28.735 INFO:tasks.rados.rados.0.smithi138.stdout:1289: write (excl) oid 31 current snap is 146 2023-06-27T12:43:28.735 INFO:tasks.rados.rados.0.smithi138.stdout:1289: seq_num 621 ranges {437147=554222,1656259=655870} 2023-06-27T12:43:28.738 INFO:tasks.rados.rados.0.smithi138.stdout:1289: writing smithi138138488-31 from 437147 to 991369 tid 1 2023-06-27T12:43:28.748 INFO:tasks.rados.rados.0.smithi138.stdout:1289: writing smithi138138488-31 from 1656259 to 2312129 tid 2 2023-06-27T12:43:28.748 INFO:tasks.rados.rados.0.smithi138.stdout:1290: snap_create 2023-06-27T12:43:28.761 INFO:tasks.rados.rados.0.smithi138.stdout:1289: finishing write tid 1 to smithi138138488-31 2023-06-27T12:43:28.812 INFO:tasks.rados.rados.0.smithi138.stdout:1289: finishing write tid 2 to smithi138138488-31 2023-06-27T12:43:28.812 INFO:tasks.rados.rados.0.smithi138.stdout:1289: finishing write tid 3 to smithi138138488-31 2023-06-27T12:43:28.813 INFO:tasks.rados.rados.0.smithi138.stdout:1289: finishing write tid 4 to smithi138138488-31 2023-06-27T12:43:28.813 INFO:tasks.rados.rados.0.smithi138.stdout:1289: oid 31 updating version 0 to 383 2023-06-27T12:43:28.813 INFO:tasks.rados.rados.0.smithi138.stdout:1289: oid 31 version 383 is already newer than 381 2023-06-27T12:43:28.813 INFO:tasks.rados.rados.0.smithi138.stdout:1289: oid 31 version 383 is already newer than 383 2023-06-27T12:43:28.813 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 383 (ObjNum 621 snap 146 seq_num 621) dirty exists 2023-06-27T12:43:28.813 INFO:tasks.rados.rados.0.smithi138.stdout:1289: left oid 31 (ObjNum 621 snap 146 seq_num 621) 2023-06-27T12:43:29.463 INFO:tasks.rados.rados.0.smithi138.stdout:1289: done (1 left) 2023-06-27T12:43:29.463 INFO:tasks.rados.rados.0.smithi138.stdout:1290: done (0 left) 2023-06-27T12:43:29.463 INFO:tasks.rados.rados.0.smithi138.stdout:1291: delete oid 40 current snap is 147 2023-06-27T12:43:29.525 INFO:tasks.rados.rados.0.smithi138.stdout:1291: done (0 left) 2023-06-27T12:43:29.525 INFO:tasks.rados.rados.0.smithi138.stdout:1292: write (excl) oid 27 current snap is 147 2023-06-27T12:43:29.525 INFO:tasks.rados.rados.0.smithi138.stdout:1292: seq_num 623 ranges {470761=704720,1656870=659539} 2023-06-27T12:43:29.531 INFO:tasks.rados.rados.0.smithi138.stdout:1292: writing smithi138138488-27 from 470761 to 1175481 tid 1 2023-06-27T12:43:29.537 INFO:tasks.rados.rados.0.smithi138.stdout:1292: writing smithi138138488-27 from 1656870 to 2316409 tid 2 2023-06-27T12:43:29.537 INFO:tasks.rados.rados.0.smithi138.stdout:1293: delete oid 49 current snap is 147 2023-06-27T12:43:29.565 INFO:tasks.rados.rados.0.smithi138.stdout:1293: done (1 left) 2023-06-27T12:43:29.565 INFO:tasks.rados.rados.0.smithi138.stdout:1294: delete oid 21 current snap is 147 2023-06-27T12:43:29.606 INFO:tasks.rados.rados.0.smithi138.stdout:1294: done (1 left) 2023-06-27T12:43:29.607 INFO:tasks.rados.rados.0.smithi138.stdout:1295: snap_create 2023-06-27T12:43:29.611 INFO:tasks.rados.rados.0.smithi138.stdout:1292: finishing write tid 1 to smithi138138488-27 2023-06-27T12:43:29.661 INFO:tasks.rados.rados.0.smithi138.stdout:1292: finishing write tid 2 to smithi138138488-27 2023-06-27T12:43:29.661 INFO:tasks.rados.rados.0.smithi138.stdout:1292: finishing write tid 3 to smithi138138488-27 2023-06-27T12:43:29.663 INFO:tasks.rados.rados.0.smithi138.stdout:1292: finishing write tid 4 to smithi138138488-27 2023-06-27T12:43:29.663 INFO:tasks.rados.rados.0.smithi138.stdout:1292: oid 27 updating version 0 to 188 2023-06-27T12:43:29.663 INFO:tasks.rados.rados.0.smithi138.stdout:1292: oid 27 version 188 is already newer than 188 2023-06-27T12:43:29.663 INFO:tasks.rados.rados.0.smithi138.stdout:1292: oid 27 version 188 is already newer than 185 2023-06-27T12:43:29.663 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 188 (ObjNum 623 snap 147 seq_num 623) dirty exists 2023-06-27T12:43:29.664 INFO:tasks.rados.rados.0.smithi138.stdout:1292: left oid 27 (ObjNum 623 snap 147 seq_num 623) 2023-06-27T12:43:30.474 INFO:tasks.rados.rados.0.smithi138.stdout:1292: done (1 left) 2023-06-27T12:43:30.474 INFO:tasks.rados.rados.0.smithi138.stdout:1295: done (0 left) 2023-06-27T12:43:30.474 INFO:tasks.rados.rados.0.smithi138.stdout:1296: snap_create 2023-06-27T12:43:30.714 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:30.853 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:30.848+0000 7fe9d2044700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:30.856 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:30.852+0000 7fe9d2044700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:30.902 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:30.904 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:30.905 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:31.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~176s 2023-06-27T12:43:31.482 INFO:tasks.rados.rados.0.smithi138.stdout:1296: done (0 left) 2023-06-27T12:43:31.482 INFO:tasks.rados.rados.0.smithi138.stdout:1297: read oid 20 snap -1 2023-06-27T12:43:31.482 INFO:tasks.rados.rados.0.smithi138.stdout:1297: expect (ObjNum 558 snap 132 seq_num 558) 2023-06-27T12:43:31.482 INFO:tasks.rados.rados.0.smithi138.stdout:1298: snap_remove snap 147 2023-06-27T12:43:32.488 INFO:tasks.rados.rados.0.smithi138.stdout:1298: done (1 left) 2023-06-27T12:43:32.489 INFO:tasks.rados.rados.0.smithi138.stdout:1299: snap_create 2023-06-27T12:43:32.489 INFO:tasks.rados.rados.0.smithi138.stdout:1297: expect (ObjNum 558 snap 132 seq_num 558) 2023-06-27T12:43:32.907 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:33.033 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:33.028+0000 7fd3c91d9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:33.035 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:33.032+0000 7fd3c91d9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:33.081 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:33.084 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:33.084 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:33.209 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:33.210 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:33.210 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:33.330+0000 7f381dc7b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:33.336+0000 7f381dc7b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:33.487 INFO:tasks.rados.rados.0.smithi138.stdout:1297: done (1 left) 2023-06-27T12:43:33.488 INFO:tasks.rados.rados.0.smithi138.stdout:1299: done (0 left) 2023-06-27T12:43:33.488 INFO:tasks.rados.rados.0.smithi138.stdout:1300: delete oid 50 current snap is 150 2023-06-27T12:43:33.543 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:33.545 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:33.556 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2023-06-27T12:43:33.482679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97635658,"num_objects":318,"num_object_clones":271,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1885,"num_read_kb":1376011,"num_write":1701,"num_write_kb":494141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3104,"ondisk_log_size":3104,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":959432,"kb_used_data":821456,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376527928,"statfs":{"total":386547056640,"available":385564598272,"internally_reserved":0,"allocated":841170944,"data_stored":951980301,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024585"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":756,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.470003+0000","last_change":"2023-06-27T12:43:28.475358+0000","last_active":"2023-06-27T12:43:28.470003+0000","last_peered":"2023-06-27T12:43:28.470003+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:28.470003+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"306'184","reported_seq":992,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.602608+0000","last_change":"2023-06-27T12:43:28.549386+0000","last_active":"2023-06-27T12:43:28.602608+0000","last_peered":"2023-06-27T12:43:28.602608+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:28.602608+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13042628,"num_objects":23,"num_object_clones":19,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":138,"num_read_kb":172784,"num_write":133,"num_write_kb":39302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"301'254","reported_seq":982,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.619648+0000","last_change":"2023-06-27T12:43:28.619648+0000","last_active":"2023-06-27T12:43:28.619648+0000","last_peered":"2023-06-27T12:43:28.619648+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:28.619648+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5462473,"num_objects":30,"num_object_clones":25,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":207,"num_read_kb":131103,"num_write":165,"num_write_kb":47622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"310'141","reported_seq":927,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.473137+0000","last_change":"2023-06-27T12:43:28.479392+0000","last_active":"2023-06-27T12:43:28.473137+0000","last_peered":"2023-06-27T12:43:28.473137+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:28.473137+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8924847,"num_objects":23,"num_object_clones":19,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":141,"num_read_kb":96601,"num_write":106,"num_write_kb":29985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":23}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"305'91","reported_seq":482,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.474527+0000","last_change":"2023-06-27T12:43:28.477876+0000","last_active":"2023-06-27T12:43:28.474527+0000","last_peered":"2023-06-27T12:43:28.474527+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:28.474527+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3848902,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":88,"num_read_kb":50182,"num_write":49,"num_write_kb":15136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"315'383","reported_seq":1020,"reported_epoch":315,"state":"active+clean","last_fresh":"2023-06-27T12:43:28.760838+0000","last_change":"2023-06-27T12:43:28.477603+0000","last_active":"2023-06-27T12:43:28.760838+0000","last_peered":"2023-06-27T12:43:28.760838+0000","last_clean":"2023-06-27T12:43:28.760838+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:28.760838+0000","last_undegraded":"2023-06-27T12:43:28.760838+0000","last_fullsized":"2023-06-27T12:43:28.760838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9798778,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":302,"num_read_kb":193212,"num_write":228,"num_write_kb":65973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"309'178","reported_seq":827,"reported_epoch":315,"state":"active+clean","last_fresh":"2023-06-27T12:43:28.478626+0000","last_change":"2023-06-27T12:43:28.478626+0000","last_active":"2023-06-27T12:43:28.478626+0000","last_peered":"2023-06-27T12:43:28.478626+0000","last_clean":"2023-06-27T12:43:28.478626+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:28.478626+0000","last_undegraded":"2023-06-27T12:43:28.478626+0000","last_fullsized":"2023-06-27T12:43:28.478626+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5966716,"num_objects":21,"num_object_clones":18,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":118,"num_read_kb":78360,"num_write":108,"num_write_kb":31422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":789,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.613148+0000","last_change":"2023-06-27T12:43:28.617624+0000","last_active":"2023-06-27T12:43:28.613148+0000","last_peered":"2023-06-27T12:43:28.613148+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:28.613148+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285851,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":526,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-06-27T12:43:28.534403+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:28.534403+0000","last_peered":"2023-06-27T12:43:28.534403+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:28.534403+0000","last_undegraded":"2023-06-27T12:43:28.534403+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":703,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-06-27T12:43:28.613145+0000","last_change":"2023-06-27T12:43:28.618801+0000","last_active":"2023-06-27T12:43:28.613145+0000","last_peered":"2023-06-27T12:43:28.613145+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:28.613145+0000","last_undegraded":"2023-06-27T12:43:28.613145+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":509,"reported_epoch":315,"state":"active+clean","last_fresh":"2023-06-27T12:43:28.472550+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:28.472550+0000","last_peered":"2023-06-27T12:43:28.472550+0000","last_clean":"2023-06-27T12:43:28.472550+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:28.472550+0000","last_undegraded":"2023-06-27T12:43:28.472550+0000","last_fullsized":"2023-06-27T12:43:28.472550+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"309'171","reported_seq":907,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.540990+0000","last_change":"2023-06-27T12:43:28.550161+0000","last_active":"2023-06-27T12:43:28.540990+0000","last_peered":"2023-06-27T12:43:28.540990+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:28.540990+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806868,"num_objects":20,"num_object_clones":15,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":72,"num_read_kb":70434,"num_write":106,"num_write_kb":29767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":492,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-06-27T12:43:28.608391+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:28.608391+0000","last_peered":"2023-06-27T12:43:28.608391+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:28.608391+0000","last_undegraded":"2023-06-27T12:43:28.608391+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"304'404","reported_seq":578,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.548644+0000","last_change":"2023-06-27T12:43:28.548644+0000","last_active":"2023-06-27T12:43:28.548644+0000","last_peered":"2023-06-27T12:43:28.548644+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:28.548644+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294403,"num_objects":21,"num_object_clones":19,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":57,"num_read_kb":33573,"num_write":108,"num_write_kb":33405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":489,"reported_epoch":315,"state":"active+clean","last_fresh":"2023-06-27T12:43:28.606875+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:28.606875+0000","last_peered":"2023-06-27T12:43:28.606875+0000","last_clean":"2023-06-27T12:43:28.606875+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:28.606875+0000","last_undegraded":"2023-06-27T12:43:28.606875+0000","last_fullsized":"2023-06-27T12:43:28.606875+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":261,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-06-27T12:43:28.474670+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:28.474670+0000","last_peered":"2023-06-27T12:43:28.474670+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:28.474670+0000","last_undegraded":"2023-06-27T12:43:28.474670+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"311'364","reported_seq":1117,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.611164+0000","last_change":"2023-06-27T12:43:28.614745+0000","last_active":"2023-06-27T12:43:28.611164+0000","last_peered":"2023-06-27T12:43:28.611164+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:28.611164+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8571933,"num_objects":47,"num_object_clones":41,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":290,"num_read_kb":173903,"num_write":265,"num_write_kb":77232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":47}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":509,"reported_epoch":315,"state":"active+clean","last_fresh":"2023-06-27T12:43:28.472237+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:28.472237+0000","last_peered":"2023-06-27T12:43:28.472237+0000","last_clean":"2023-06-27T12:43:28.472237+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:28.472237+0000","last_undegraded":"2023-06-27T12:43:28.472237+0000","last_fullsized":"2023-06-27T12:43:28.472237+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"315'212","reported_seq":501,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.649911+0000","last_change":"2023-06-27T12:43:28.619229+0000","last_active":"2023-06-27T12:43:28.649911+0000","last_peered":"2023-06-27T12:43:28.649911+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:28.649911+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6756843,"num_objects":13,"num_object_clones":13,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":53232,"num_write":75,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":13}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":269,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-06-27T12:43:28.541968+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:28.541968+0000","last_peered":"2023-06-27T12:43:28.541968+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:28.541968+0000","last_undegraded":"2023-06-27T12:43:28.541968+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"314'186","reported_seq":945,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.540038+0000","last_change":"2023-06-27T12:43:28.599807+0000","last_active":"2023-06-27T12:43:28.540038+0000","last_peered":"2023-06-27T12:43:28.540038+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:28.540038+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9950239,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":109559,"num_write":126,"num_write_kb":36403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":24}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"310'115","reported_seq":474,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.608872+0000","last_change":"2023-06-27T12:43:28.614324+0000","last_active":"2023-06-27T12:43:28.608872+0000","last_peered":"2023-06-27T12:43:28.608872+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:28.608872+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933102,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":53577,"num_write":43,"num_write_kb":12009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":526,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-06-27T12:43:28.532858+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:28.532858+0000","last_peered":"2023-06-27T12:43:28.532858+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:28.532858+0000","last_undegraded":"2023-06-27T12:43:28.532858+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":489,"reported_epoch":315,"state":"active+clean","last_fresh":"2023-06-27T12:43:28.610666+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:28.610666+0000","last_peered":"2023-06-27T12:43:28.610666+0000","last_clean":"2023-06-27T12:43:28.610666+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:28.610666+0000","last_undegraded":"2023-06-27T12:43:28.610666+0000","last_fullsized":"2023-06-27T12:43:28.610666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"305'220","reported_seq":955,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:28.539871+0000","last_change":"2023-06-27T12:43:28.545317+0000","last_active":"2023-06-27T12:43:28.539871+0000","last_peered":"2023-06-27T12:43:28.539871+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:28.539871+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832260,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":112358,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":97635658,"num_objects":318,"num_object_clones":271,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1885,"num_read_kb":1376011,"num_write":1701,"num_write_kb":494141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3007,"ondisk_log_size":3007,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":246632,"kb_used_data":211132,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94125208,"statfs":{"total":96636764160,"available":96384212992,"internally_reserved":0,"allocated":216199168,"data_stored":244645440,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":253012,"kb_used_data":216956,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94118828,"statfs":{"total":96636764160,"available":96377679872,"internally_reserved":0,"allocated":222162944,"data_stored":248741241,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":278508,"kb_used_data":243008,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94093332,"statfs":{"total":96636764160,"available":96351571968,"internally_reserved":0,"allocated":248840192,"data_stored":281554068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:33.558 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.c 3.3 3.2 2023-06-27T12:43:33.562 INFO:tasks.rados.rados.0.smithi138.stdout:1300: done (0 left) 2023-06-27T12:43:33.563 INFO:tasks.rados.rados.0.smithi138.stdout:1301: write (excl) oid 1 current snap is 150 2023-06-27T12:43:33.563 INFO:tasks.rados.rados.0.smithi138.stdout:1301: seq_num 627 ranges {537989=722069,1890798=647191} 2023-06-27T12:43:33.570 INFO:tasks.rados.rados.0.smithi138.stdout:1301: writing smithi138138488-1 from 537989 to 1260058 tid 1 2023-06-27T12:43:33.580 INFO:tasks.rados.rados.0.smithi138.stdout:1301: writing smithi138138488-1 from 1890798 to 2537989 tid 2 2023-06-27T12:43:33.580 INFO:tasks.rados.rados.0.smithi138.stdout:1302: snap_create 2023-06-27T12:43:33.601 INFO:tasks.rados.rados.0.smithi138.stdout:1301: finishing write tid 1 to smithi138138488-1 2023-06-27T12:43:33.641 INFO:tasks.rados.rados.0.smithi138.stdout:1301: finishing write tid 2 to smithi138138488-1 2023-06-27T12:43:33.660 INFO:tasks.rados.rados.0.smithi138.stdout:1301: finishing write tid 3 to smithi138138488-1 2023-06-27T12:43:33.661 INFO:tasks.rados.rados.0.smithi138.stdout:1301: finishing write tid 4 to smithi138138488-1 2023-06-27T12:43:33.662 INFO:tasks.rados.rados.0.smithi138.stdout:1301: oid 1 updating version 0 to 370 2023-06-27T12:43:33.662 INFO:tasks.rados.rados.0.smithi138.stdout:1301: oid 1 version 370 is already newer than 367 2023-06-27T12:43:33.662 INFO:tasks.rados.rados.0.smithi138.stdout:1301: oid 1 version 370 is already newer than 370 2023-06-27T12:43:33.662 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 370 (ObjNum 627 snap 150 seq_num 627) dirty exists 2023-06-27T12:43:33.662 INFO:tasks.rados.rados.0.smithi138.stdout:1301: left oid 1 (ObjNum 627 snap 150 seq_num 627) 2023-06-27T12:43:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:33.690+0000 7f0899e5c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:33.700+0000 7f0899e5c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:33.898 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c,3.f] on osd.0 to force-backfill; instructing pg(s) [3.2] on osd.1 to force-backfill; instructing pg(s) [3.3] on osd.2 to force-backfill; 2023-06-27T12:43:33.898 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:34.495 INFO:tasks.rados.rados.0.smithi138.stdout:1301: done (1 left) 2023-06-27T12:43:34.495 INFO:tasks.rados.rados.0.smithi138.stdout:1302: done (0 left) 2023-06-27T12:43:34.496 INFO:tasks.rados.rados.0.smithi138.stdout:1303: write (excl) oid 33 current snap is 151 2023-06-27T12:43:34.496 INFO:tasks.rados.rados.0.smithi138.stdout:1303: seq_num 628 ranges {554796=797318,1949280=526888} 2023-06-27T12:43:34.505 INFO:tasks.rados.rados.0.smithi138.stdout:1303: writing smithi138138488-33 from 554796 to 1352114 tid 1 2023-06-27T12:43:34.513 INFO:tasks.rados.rados.0.smithi138.stdout:1303: writing smithi138138488-33 from 1949280 to 2476168 tid 2 2023-06-27T12:43:34.513 INFO:tasks.rados.rados.0.smithi138.stdout:1304: snap_remove snap 146 2023-06-27T12:43:35.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:35.212 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:35.207+0000 7f9aa6f3c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:35.215 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:35.211+0000 7f9aa6f3c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:35.261 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:35.263 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:35.264 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:35.449 INFO:tasks.rados.rados.0.smithi138.stdout:1304: done (1 left) 2023-06-27T12:43:35.449 INFO:tasks.rados.rados.0.smithi138.stdout:1305: delete oid 41 current snap is 151 2023-06-27T12:43:35.450 INFO:tasks.rados.rados.0.smithi138.stdout:1303: finishing write tid 1 to smithi138138488-33 2023-06-27T12:43:35.450 INFO:tasks.rados.rados.0.smithi138.stdout:1303: finishing write tid 2 to smithi138138488-33 2023-06-27T12:43:35.450 INFO:tasks.rados.rados.0.smithi138.stdout:1303: finishing write tid 3 to smithi138138488-33 2023-06-27T12:43:35.450 INFO:tasks.rados.rados.0.smithi138.stdout:1303: finishing write tid 4 to smithi138138488-33 2023-06-27T12:43:35.450 INFO:tasks.rados.rados.0.smithi138.stdout:1303: oid 33 updating version 0 to 151 2023-06-27T12:43:35.450 INFO:tasks.rados.rados.0.smithi138.stdout:1303: oid 33 version 151 is already newer than 151 2023-06-27T12:43:35.450 INFO:tasks.rados.rados.0.smithi138.stdout:1303: oid 33 version 151 is already newer than 148 2023-06-27T12:43:35.450 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 151 (ObjNum 628 snap 151 seq_num 628) dirty exists 2023-06-27T12:43:35.451 INFO:tasks.rados.rados.0.smithi138.stdout:1303: left oid 33 (ObjNum 628 snap 151 seq_num 628) 2023-06-27T12:43:35.527 INFO:tasks.rados.rados.0.smithi138.stdout:1303: done (1 left) 2023-06-27T12:43:35.527 INFO:tasks.rados.rados.0.smithi138.stdout:1305: done (0 left) 2023-06-27T12:43:35.527 INFO:tasks.rados.rados.0.smithi138.stdout:1306: snap_create 2023-06-27T12:43:36.500 INFO:tasks.rados.rados.0.smithi138.stdout:1306: done (0 left) 2023-06-27T12:43:36.501 INFO:tasks.rados.rados.0.smithi138.stdout:1307: delete oid 46 current snap is 152 2023-06-27T12:43:36.528 INFO:tasks.rados.rados.0.smithi138.stdout:1307: done (0 left) 2023-06-27T12:43:36.528 INFO:tasks.rados.rados.0.smithi138.stdout:1308: delete oid 16 current snap is 152 2023-06-27T12:43:36.564 INFO:tasks.rados.rados.0.smithi138.stdout:1308: done (0 left) 2023-06-27T12:43:36.564 INFO:tasks.rados.rados.0.smithi138.stdout:1309: write (excl) oid 24 current snap is 152 2023-06-27T12:43:36.564 INFO:tasks.rados.rados.0.smithi138.stdout:1309: seq_num 632 ranges {622024=414667,1783208=417873} 2023-06-27T12:43:36.569 INFO:tasks.rados.rados.0.smithi138.stdout:1309: writing smithi138138488-24 from 622024 to 1036691 tid 1 2023-06-27T12:43:36.577 INFO:tasks.rados.rados.0.smithi138.stdout:1309: writing smithi138138488-24 from 1783208 to 2201081 tid 2 2023-06-27T12:43:36.577 INFO:tasks.rados.rados.0.smithi138.stdout:1310: read oid 40 snap -1 2023-06-27T12:43:36.577 INFO:tasks.rados.rados.0.smithi138.stdout:1310: expect deleted 2023-06-27T12:43:36.577 INFO:tasks.rados.rados.0.smithi138.stdout:1311: write oid 30 current snap is 152 2023-06-27T12:43:36.577 INFO:tasks.rados.rados.0.smithi138.stdout:1311: seq_num 633 ranges {638831=489916,1841690=677884} 2023-06-27T12:43:36.583 INFO:tasks.rados.rados.0.smithi138.stdout:1311: writing smithi138138488-30 from 638831 to 1128747 tid 1 2023-06-27T12:43:36.591 INFO:tasks.rados.rados.0.smithi138.stdout:1311: writing smithi138138488-30 from 1841690 to 2519574 tid 2 2023-06-27T12:43:36.591 INFO:tasks.rados.rados.0.smithi138.stdout:1312: write (excl) oid 23 current snap is 152 2023-06-27T12:43:36.592 INFO:tasks.rados.rados.0.smithi138.stdout:1312: seq_num 634 ranges {655638=565165,1900172=421542} 2023-06-27T12:43:36.600 INFO:tasks.rados.rados.0.smithi138.stdout:1312: writing smithi138138488-23 from 655638 to 1220803 tid 1 2023-06-27T12:43:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:1312: writing smithi138138488-23 from 1900172 to 2321714 tid 2 2023-06-27T12:43:36.606 INFO:tasks.rados.rados.0.smithi138.stdout:1313: snap_create 2023-06-27T12:43:36.637 INFO:tasks.rados.rados.0.smithi138.stdout:1312: finishing write tid 1 to smithi138138488-23 2023-06-27T12:43:36.657 INFO:tasks.rados.rados.0.smithi138.stdout:1309: finishing write tid 1 to smithi138138488-24 2023-06-27T12:43:36.680 INFO:tasks.rados.rados.0.smithi138.stdout:1311: finishing write tid 1 to smithi138138488-30 2023-06-27T12:43:36.684 INFO:tasks.rados.rados.0.smithi138.stdout:1311: finishing write tid 2 to smithi138138488-30 2023-06-27T12:43:36.685 INFO:tasks.rados.rados.0.smithi138.stdout:1309: finishing write tid 2 to smithi138138488-24 2023-06-27T12:43:36.686 INFO:tasks.rados.rados.0.smithi138.stdout:1309: finishing write tid 3 to smithi138138488-24 2023-06-27T12:43:36.687 INFO:tasks.rados.rados.0.smithi138.stdout:1309: finishing write tid 4 to smithi138138488-24 2023-06-27T12:43:36.687 INFO:tasks.rados.rados.0.smithi138.stdout:1309: oid 24 updating version 0 to 389 2023-06-27T12:43:36.687 INFO:tasks.rados.rados.0.smithi138.stdout:1309: oid 24 version 389 is already newer than 389 2023-06-27T12:43:36.687 INFO:tasks.rados.rados.0.smithi138.stdout:1309: oid 24 version 389 is already newer than 386 2023-06-27T12:43:36.688 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 389 (ObjNum 632 snap 152 seq_num 632) dirty exists 2023-06-27T12:43:36.688 INFO:tasks.rados.rados.0.smithi138.stdout:1309: left oid 24 (ObjNum 632 snap 152 seq_num 632) 2023-06-27T12:43:36.696 INFO:tasks.rados.rados.0.smithi138.stdout:1312: finishing write tid 2 to smithi138138488-23 2023-06-27T12:43:36.697 INFO:tasks.rados.rados.0.smithi138.stdout:1312: finishing write tid 3 to smithi138138488-23 2023-06-27T12:43:36.698 INFO:tasks.rados.rados.0.smithi138.stdout:1312: finishing write tid 4 to smithi138138488-23 2023-06-27T12:43:36.699 INFO:tasks.rados.rados.0.smithi138.stdout:1312: oid 23 updating version 0 to 175 2023-06-27T12:43:36.699 INFO:tasks.rados.rados.0.smithi138.stdout:1312: oid 23 version 175 is already newer than 173 2023-06-27T12:43:36.699 INFO:tasks.rados.rados.0.smithi138.stdout:1312: oid 23 version 175 is already newer than 175 2023-06-27T12:43:36.699 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 175 (ObjNum 634 snap 152 seq_num 634) dirty exists 2023-06-27T12:43:36.700 INFO:tasks.rados.rados.0.smithi138.stdout:1312: left oid 23 (ObjNum 634 snap 152 seq_num 634) 2023-06-27T12:43:36.725 INFO:tasks.rados.rados.0.smithi138.stdout:1311: finishing write tid 3 to smithi138138488-30 2023-06-27T12:43:36.726 INFO:tasks.rados.rados.0.smithi138.stdout:1311: finishing write tid 4 to smithi138138488-30 2023-06-27T12:43:36.726 INFO:tasks.rados.rados.0.smithi138.stdout:1311: oid 30 updating version 0 to 216 2023-06-27T12:43:36.727 INFO:tasks.rados.rados.0.smithi138.stdout:1311: oid 30 version 216 is already newer than 216 2023-06-27T12:43:36.727 INFO:tasks.rados.rados.0.smithi138.stdout:1311: oid 30 version 216 is already newer than 216 2023-06-27T12:43:36.727 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 216 (ObjNum 633 snap 152 seq_num 633) dirty exists 2023-06-27T12:43:36.727 INFO:tasks.rados.rados.0.smithi138.stdout:1311: left oid 30 (ObjNum 633 snap 152 seq_num 633) 2023-06-27T12:43:36.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~182s 2023-06-27T12:43:37.266 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:37.393 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:37.388+0000 7fc26cb8d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:37.396 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:37.392+0000 7fc26cb8d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:37.441 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:37.444 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:37.444 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:37.503 INFO:tasks.rados.rados.0.smithi138.stdout:1309: done (4 left) 2023-06-27T12:43:37.504 INFO:tasks.rados.rados.0.smithi138.stdout:1310: done (3 left) 2023-06-27T12:43:37.504 INFO:tasks.rados.rados.0.smithi138.stdout:1311: done (2 left) 2023-06-27T12:43:37.504 INFO:tasks.rados.rados.0.smithi138.stdout:1312: done (1 left) 2023-06-27T12:43:37.504 INFO:tasks.rados.rados.0.smithi138.stdout:1313: done (0 left) 2023-06-27T12:43:37.504 INFO:tasks.rados.rados.0.smithi138.stdout:1314: read oid 1 snap 149 2023-06-27T12:43:37.504 INFO:tasks.rados.rados.0.smithi138.stdout:1314: expect (ObjNum 611 snap 145 seq_num 611) 2023-06-27T12:43:37.505 INFO:tasks.rados.rados.0.smithi138.stdout:1315: snap_remove snap 148 2023-06-27T12:43:38.508 INFO:tasks.rados.rados.0.smithi138.stdout:1315: done (1 left) 2023-06-27T12:43:38.508 INFO:tasks.rados.rados.0.smithi138.stdout:1316: write oid 39 current snap is 153 2023-06-27T12:43:38.508 INFO:tasks.rados.rados.0.smithi138.stdout:1316: seq_num 635 ranges {672445=640414,1842301=681553} 2023-06-27T12:43:38.514 INFO:tasks.rados.rados.0.smithi138.stdout:1316: writing smithi138138488-39 from 672445 to 1312859 tid 1 2023-06-27T12:43:38.521 INFO:tasks.rados.rados.0.smithi138.stdout:1316: writing smithi138138488-39 from 1842301 to 2523854 tid 2 2023-06-27T12:43:38.522 INFO:tasks.rados.rados.0.smithi138.stdout:1317: delete oid 34 current snap is 153 2023-06-27T12:43:38.522 INFO:tasks.rados.rados.0.smithi138.stdout:1314: expect (ObjNum 611 snap 145 seq_num 611) 2023-06-27T12:43:38.628 INFO:tasks.rados.rados.0.smithi138.stdout:1314: done (2 left) 2023-06-27T12:43:38.628 INFO:tasks.rados.rados.0.smithi138.stdout:1317: done (1 left) 2023-06-27T12:43:38.628 INFO:tasks.rados.rados.0.smithi138.stdout:1318: write (excl) oid 10 current snap is 153 2023-06-27T12:43:38.628 INFO:tasks.rados.rados.0.smithi138.stdout:1318: seq_num 637 ranges {706059=790912,1959265=685222} 2023-06-27T12:43:38.636 INFO:tasks.rados.rados.0.smithi138.stdout:1318: writing smithi138138488-10 from 706059 to 1496971 tid 1 2023-06-27T12:43:38.643 INFO:tasks.rados.rados.0.smithi138.stdout:1318: writing smithi138138488-10 from 1959265 to 2644487 tid 2 2023-06-27T12:43:38.644 INFO:tasks.rados.rados.0.smithi138.stdout:1319: read oid 28 snap -1 2023-06-27T12:43:38.644 INFO:tasks.rados.rados.0.smithi138.stdout:1319: expect deleted 2023-06-27T12:43:38.644 INFO:tasks.rados.rados.0.smithi138.stdout:1320: write (excl) oid 3 current snap is 153 2023-06-27T12:43:38.644 INFO:tasks.rados.rados.0.smithi138.stdout:1320: seq_num 638 ranges {722866=466161,1617747=428880,2587634=135232} 2023-06-27T12:43:38.650 INFO:tasks.rados.rados.0.smithi138.stdout:1320: writing smithi138138488-3 from 722866 to 1189027 tid 1 2023-06-27T12:43:38.655 INFO:tasks.rados.rados.0.smithi138.stdout:1320: writing smithi138138488-3 from 1617747 to 2046627 tid 2 2023-06-27T12:43:38.659 INFO:tasks.rados.rados.0.smithi138.stdout:1320: writing smithi138138488-3 from 2587634 to 2722866 tid 3 2023-06-27T12:43:38.659 INFO:tasks.rados.rados.0.smithi138.stdout:1321: write oid 36 current snap is 153 2023-06-27T12:43:38.659 INFO:tasks.rados.rados.0.smithi138.stdout:1316: finishing write tid 1 to smithi138138488-39 2023-06-27T12:43:38.659 INFO:tasks.rados.rados.0.smithi138.stdout:1321: seq_num 639 ranges {739673=657763,2076229=663444} 2023-06-27T12:43:38.667 INFO:tasks.rados.rados.0.smithi138.stdout:1321: writing smithi138138488-36 from 739673 to 1397436 tid 1 2023-06-27T12:43:38.675 INFO:tasks.rados.rados.0.smithi138.stdout:1321: writing smithi138138488-36 from 2076229 to 2739673 tid 2 2023-06-27T12:43:38.675 INFO:tasks.rados.rados.0.smithi138.stdout:1322: read oid 33 snap -1 2023-06-27T12:43:38.675 INFO:tasks.rados.rados.0.smithi138.stdout:1322: expect (ObjNum 628 snap 151 seq_num 628) 2023-06-27T12:43:38.675 INFO:tasks.rados.rados.0.smithi138.stdout:1323: snap_remove snap 152 2023-06-27T12:43:38.911 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:38.911 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:38.911 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:39.033+0000 7f107b045700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:39.039+0000 7f107b045700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:39.236 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:39.250 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2023-06-27T12:43:38.502690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":91450319,"num_objects":323,"num_object_clones":276,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1895,"num_read_kb":1380719,"num_write":1709,"num_write_kb":496814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3130,"ondisk_log_size":3130,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":962912,"kb_used_data":826944,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376524448,"statfs":{"total":386547056640,"available":385561034752,"internally_reserved":0,"allocated":846790656,"data_stored":958355804,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013142"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":765,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.809537+0000","last_change":"2023-06-27T12:43:32.526856+0000","last_active":"2023-06-27T12:43:33.809537+0000","last_peered":"2023-06-27T12:43:33.809537+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:33.809537+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"316'188","reported_seq":1007,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.494787+0000","last_change":"2023-06-27T12:43:32.508237+0000","last_active":"2023-06-27T12:43:33.494787+0000","last_peered":"2023-06-27T12:43:33.494787+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:33.494787+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":177491,"num_write":137,"num_write_kb":40636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":24}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"301'254","reported_seq":993,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.498490+0000","last_change":"2023-06-27T12:43:32.562323+0000","last_active":"2023-06-27T12:43:33.498490+0000","last_peered":"2023-06-27T12:43:33.498490+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:33.498490+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5462473,"num_objects":30,"num_object_clones":25,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":207,"num_read_kb":131103,"num_write":165,"num_write_kb":47622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"316'147","reported_seq":937,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.814858+0000","last_change":"2023-06-27T12:43:32.552830+0000","last_active":"2023-06-27T12:43:33.814858+0000","last_peered":"2023-06-27T12:43:33.814858+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:33.814858+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6739805,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":96601,"num_write":106,"num_write_kb":29985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"305'91","reported_seq":491,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.811813+0000","last_change":"2023-06-27T12:43:32.543138+0000","last_active":"2023-06-27T12:43:33.811813+0000","last_peered":"2023-06-27T12:43:33.811813+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:33.811813+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3848902,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":88,"num_read_kb":50182,"num_write":49,"num_write_kb":15136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"315'383","reported_seq":1029,"reported_epoch":320,"state":"active+clean","last_fresh":"2023-06-27T12:43:33.807051+0000","last_change":"2023-06-27T12:43:32.544082+0000","last_active":"2023-06-27T12:43:33.807051+0000","last_peered":"2023-06-27T12:43:33.807051+0000","last_clean":"2023-06-27T12:43:33.807051+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:33.807051+0000","last_undegraded":"2023-06-27T12:43:33.807051+0000","last_fullsized":"2023-06-27T12:43:33.807051+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9798778,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":302,"num_read_kb":193212,"num_write":228,"num_write_kb":65973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"315'179","reported_seq":836,"reported_epoch":320,"state":"active+clean","last_fresh":"2023-06-27T12:43:33.808789+0000","last_change":"2023-06-27T12:43:32.554192+0000","last_active":"2023-06-27T12:43:33.808789+0000","last_peered":"2023-06-27T12:43:33.808789+0000","last_clean":"2023-06-27T12:43:33.808789+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:33.808789+0000","last_undegraded":"2023-06-27T12:43:33.808789+0000","last_fullsized":"2023-06-27T12:43:33.808789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5966716,"num_objects":21,"num_object_clones":18,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":118,"num_read_kb":78360,"num_write":108,"num_write_kb":31422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"249'67","reported_seq":801,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.498100+0000","last_change":"2023-06-27T12:43:32.560594+0000","last_active":"2023-06-27T12:43:33.498100+0000","last_peered":"2023-06-27T12:43:33.498100+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:33.498100+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":64,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":534,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-06-27T12:43:33.494238+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:33.494238+0000","last_peered":"2023-06-27T12:43:33.494238+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:33.494238+0000","last_undegraded":"2023-06-27T12:43:33.494238+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":714,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-06-27T12:43:33.499296+0000","last_change":"2023-06-27T12:43:32.563585+0000","last_active":"2023-06-27T12:43:33.499296+0000","last_peered":"2023-06-27T12:43:33.499296+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:33.499296+0000","last_undegraded":"2023-06-27T12:43:33.499296+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":517,"reported_epoch":320,"state":"active+clean","last_fresh":"2023-06-27T12:43:33.807042+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:33.807042+0000","last_peered":"2023-06-27T12:43:33.807042+0000","last_clean":"2023-06-27T12:43:33.807042+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:33.807042+0000","last_undegraded":"2023-06-27T12:43:33.807042+0000","last_fullsized":"2023-06-27T12:43:33.807042+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"309'171","reported_seq":916,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.503037+0000","last_change":"2023-06-27T12:43:32.507711+0000","last_active":"2023-06-27T12:43:33.503037+0000","last_peered":"2023-06-27T12:43:33.503037+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:33.503037+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806868,"num_objects":20,"num_object_clones":15,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":72,"num_read_kb":70434,"num_write":106,"num_write_kb":29767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":502,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-06-27T12:43:33.502281+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:33.502281+0000","last_peered":"2023-06-27T12:43:33.502281+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:33.502281+0000","last_undegraded":"2023-06-27T12:43:33.502281+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"315'413","reported_seq":587,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.504112+0000","last_change":"2023-06-27T12:43:32.506329+0000","last_active":"2023-06-27T12:43:33.504112+0000","last_peered":"2023-06-27T12:43:33.504112+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:33.504112+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294403,"num_objects":21,"num_object_clones":19,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":57,"num_read_kb":33573,"num_write":108,"num_write_kb":33405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":499,"reported_epoch":320,"state":"active+clean","last_fresh":"2023-06-27T12:43:33.509567+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:33.509567+0000","last_peered":"2023-06-27T12:43:33.509567+0000","last_clean":"2023-06-27T12:43:33.509567+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:33.509567+0000","last_undegraded":"2023-06-27T12:43:33.509567+0000","last_fullsized":"2023-06-27T12:43:33.509567+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":269,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-06-27T12:43:33.810093+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:33.810093+0000","last_peered":"2023-06-27T12:43:33.810093+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:33.810093+0000","last_undegraded":"2023-06-27T12:43:33.810093+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"320'370","reported_seq":1133,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.604173+0000","last_change":"2023-06-27T12:43:32.550510+0000","last_active":"2023-06-27T12:43:33.604173+0000","last_peered":"2023-06-27T12:43:33.604173+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:33.604173+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521347,"num_objects":49,"num_object_clones":43,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":293,"num_read_kb":173904,"num_write":269,"num_write_kb":78571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":49}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":517,"reported_epoch":320,"state":"active+clean","last_fresh":"2023-06-27T12:43:33.811070+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:33.811070+0000","last_peered":"2023-06-27T12:43:33.811070+0000","last_clean":"2023-06-27T12:43:33.811070+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:33.811070+0000","last_undegraded":"2023-06-27T12:43:33.811070+0000","last_fullsized":"2023-06-27T12:43:33.811070+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"315'212","reported_seq":512,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.497643+0000","last_change":"2023-06-27T12:43:32.563135+0000","last_active":"2023-06-27T12:43:33.497643+0000","last_peered":"2023-06-27T12:43:33.497643+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:33.497643+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6756843,"num_objects":13,"num_object_clones":13,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":53232,"num_write":75,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":13}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":277,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-06-27T12:43:33.500104+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:33.500104+0000","last_peered":"2023-06-27T12:43:33.500104+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:33.500104+0000","last_undegraded":"2023-06-27T12:43:33.500104+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"314'186","reported_seq":954,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.500778+0000","last_change":"2023-06-27T12:43:32.504002+0000","last_active":"2023-06-27T12:43:33.500778+0000","last_peered":"2023-06-27T12:43:33.500778+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:33.500778+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9950239,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":109559,"num_write":126,"num_write_kb":36403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":24}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"310'115","reported_seq":485,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.507693+0000","last_change":"2023-06-27T12:43:32.550919+0000","last_active":"2023-06-27T12:43:33.507693+0000","last_peered":"2023-06-27T12:43:33.507693+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:33.507693+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933102,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":53577,"num_write":43,"num_write_kb":12009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":534,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-06-27T12:43:33.498018+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:33.498018+0000","last_peered":"2023-06-27T12:43:33.498018+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:33.498018+0000","last_undegraded":"2023-06-27T12:43:33.498018+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":499,"reported_epoch":320,"state":"active+clean","last_fresh":"2023-06-27T12:43:33.497046+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:33.497046+0000","last_peered":"2023-06-27T12:43:33.497046+0000","last_clean":"2023-06-27T12:43:33.497046+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:33.497046+0000","last_undegraded":"2023-06-27T12:43:33.497046+0000","last_fullsized":"2023-06-27T12:43:33.497046+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"305'220","reported_seq":965,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:33.495432+0000","last_change":"2023-06-27T12:43:32.503715+0000","last_active":"2023-06-27T12:43:33.495432+0000","last_peered":"2023-06-27T12:43:33.495432+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:33.495432+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832260,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":111,"num_read_kb":112358,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":91450319,"num_objects":323,"num_object_clones":276,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1895,"num_read_kb":1380719,"num_write":1709,"num_write_kb":496814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3033,"ondisk_log_size":3033,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673300,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":247528,"kb_used_data":212512,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94124312,"statfs":{"total":96636764160,"available":96383295488,"internally_reserved":0,"allocated":217612288,"data_stored":246150263,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":253356,"kb_used_data":218340,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94118484,"statfs":{"total":96636764160,"available":96377327616,"internally_reserved":0,"allocated":223580160,"data_stored":250434832,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673300,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":280748,"kb_used_data":245732,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94091092,"statfs":{"total":96636764160,"available":96349278208,"internally_reserved":0,"allocated":251629568,"data_stored":284731157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:39.252 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.e 3.c 3.8 3.2 3.4 3.6 2023-06-27T12:43:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:39.371+0000 7f412e492700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:39.388 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:39.384+0000 7f412e492700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:39.445 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:39.513 INFO:tasks.rados.rados.0.smithi138.stdout:1323: done (6 left) 2023-06-27T12:43:39.514 INFO:tasks.rados.rados.0.smithi138.stdout:1324: snap_create 2023-06-27T12:43:39.514 INFO:tasks.rados.rados.0.smithi138.stdout:1316: finishing write tid 2 to smithi138138488-39 2023-06-27T12:43:39.514 INFO:tasks.rados.rados.0.smithi138.stdout:1316: finishing write tid 3 to smithi138138488-39 2023-06-27T12:43:39.515 INFO:tasks.rados.rados.0.smithi138.stdout:1318: finishing write tid 1 to smithi138138488-10 2023-06-27T12:43:39.515 INFO:tasks.rados.rados.0.smithi138.stdout:1320: finishing write tid 1 to smithi138138488-3 2023-06-27T12:43:39.515 INFO:tasks.rados.rados.0.smithi138.stdout:1316: finishing write tid 4 to smithi138138488-39 2023-06-27T12:43:39.515 INFO:tasks.rados.rados.0.smithi138.stdout:1316: oid 39 updating version 0 to 222 2023-06-27T12:43:39.516 INFO:tasks.rados.rados.0.smithi138.stdout:1316: oid 39 version 222 is already newer than 222 2023-06-27T12:43:39.516 INFO:tasks.rados.rados.0.smithi138.stdout:1316: oid 39 version 222 is already newer than 222 2023-06-27T12:43:39.516 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 39 v 222 (ObjNum 635 snap 153 seq_num 635) dirty exists 2023-06-27T12:43:39.517 INFO:tasks.rados.rados.0.smithi138.stdout:1316: left oid 39 (ObjNum 635 snap 153 seq_num 635) 2023-06-27T12:43:39.517 INFO:tasks.rados.rados.0.smithi138.stdout:1321: finishing write tid 1 to smithi138138488-36 2023-06-27T12:43:39.517 INFO:tasks.rados.rados.0.smithi138.stdout:1318: finishing write tid 2 to smithi138138488-10 2023-06-27T12:43:39.517 INFO:tasks.rados.rados.0.smithi138.stdout:1318: finishing write tid 3 to smithi138138488-10 2023-06-27T12:43:39.518 INFO:tasks.rados.rados.0.smithi138.stdout:1322: expect (ObjNum 628 snap 151 seq_num 628) 2023-06-27T12:43:39.574 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:39.569+0000 7f7658bde700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:39.577 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:39.574+0000 7f7658bde700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:39.602 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c] on osd.0 to force-recovery; instructing pg(s) [3.2,3.8] on osd.1 to force-recovery; instructing pg(s) [3.4,3.6,3.e] on osd.2 to force-recovery; 2023-06-27T12:43:39.602 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:39.635 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:39.638 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:39.639 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:39.741 INFO:tasks.rados.rados.0.smithi138.stdout:1318: finishing write tid 4 to smithi138138488-10 2023-06-27T12:43:39.741 INFO:tasks.rados.rados.0.smithi138.stdout:1318: oid 10 updating version 0 to 183 2023-06-27T12:43:39.742 INFO:tasks.rados.rados.0.smithi138.stdout:1318: oid 10 version 183 is already newer than 183 2023-06-27T12:43:39.742 INFO:tasks.rados.rados.0.smithi138.stdout:1318: oid 10 version 183 is already newer than 180 2023-06-27T12:43:39.742 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 183 (ObjNum 637 snap 153 seq_num 637) dirty exists 2023-06-27T12:43:39.743 INFO:tasks.rados.rados.0.smithi138.stdout:1318: left oid 10 (ObjNum 637 snap 153 seq_num 637) 2023-06-27T12:43:39.743 INFO:tasks.rados.rados.0.smithi138.stdout:1320: finishing write tid 2 to smithi138138488-3 2023-06-27T12:43:39.743 INFO:tasks.rados.rados.0.smithi138.stdout:1320: finishing write tid 3 to smithi138138488-3 2023-06-27T12:43:39.743 INFO:tasks.rados.rados.0.smithi138.stdout:1320: finishing write tid 4 to smithi138138488-3 2023-06-27T12:43:39.744 INFO:tasks.rados.rados.0.smithi138.stdout:1320: finishing write tid 5 to smithi138138488-3 2023-06-27T12:43:39.744 INFO:tasks.rados.rados.0.smithi138.stdout:1320: oid 3 updating version 0 to 193 2023-06-27T12:43:39.744 INFO:tasks.rados.rados.0.smithi138.stdout:1320: oid 3 updating version 193 to 197 2023-06-27T12:43:39.745 INFO:tasks.rados.rados.0.smithi138.stdout:1320: oid 3 version 197 is already newer than 197 2023-06-27T12:43:39.745 INFO:tasks.rados.rados.0.smithi138.stdout:1320: oid 3 version 197 is already newer than 197 2023-06-27T12:43:39.745 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 197 (ObjNum 638 snap 153 seq_num 638) dirty exists 2023-06-27T12:43:39.745 INFO:tasks.rados.rados.0.smithi138.stdout:1320: left oid 3 (ObjNum 638 snap 153 seq_num 638) 2023-06-27T12:43:39.746 INFO:tasks.rados.rados.0.smithi138.stdout:1321: finishing write tid 2 to smithi138138488-36 2023-06-27T12:43:39.746 INFO:tasks.rados.rados.0.smithi138.stdout:1321: finishing write tid 3 to smithi138138488-36 2023-06-27T12:43:39.746 INFO:tasks.rados.rados.0.smithi138.stdout:1321: finishing write tid 4 to smithi138138488-36 2023-06-27T12:43:39.746 INFO:tasks.rados.rados.0.smithi138.stdout:1321: oid 36 updating version 0 to 268 2023-06-27T12:43:39.747 INFO:tasks.rados.rados.0.smithi138.stdout:1321: oid 36 version 268 is already newer than 268 2023-06-27T12:43:39.747 INFO:tasks.rados.rados.0.smithi138.stdout:1321: oid 36 version 268 is already newer than 268 2023-06-27T12:43:39.747 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 268 (ObjNum 639 snap 153 seq_num 639) dirty exists 2023-06-27T12:43:39.747 INFO:tasks.rados.rados.0.smithi138.stdout:1321: left oid 36 (ObjNum 639 snap 153 seq_num 639) 2023-06-27T12:43:40.518 INFO:tasks.rados.rados.0.smithi138.stdout:1316: done (6 left) 2023-06-27T12:43:40.519 INFO:tasks.rados.rados.0.smithi138.stdout:1318: done (5 left) 2023-06-27T12:43:40.519 INFO:tasks.rados.rados.0.smithi138.stdout:1319: done (4 left) 2023-06-27T12:43:40.519 INFO:tasks.rados.rados.0.smithi138.stdout:1320: done (3 left) 2023-06-27T12:43:40.519 INFO:tasks.rados.rados.0.smithi138.stdout:1321: done (2 left) 2023-06-27T12:43:40.519 INFO:tasks.rados.rados.0.smithi138.stdout:1322: done (1 left) 2023-06-27T12:43:40.519 INFO:tasks.rados.rados.0.smithi138.stdout:1324: done (0 left) 2023-06-27T12:43:40.520 INFO:tasks.rados.rados.0.smithi138.stdout:1325: read oid 22 snap -1 2023-06-27T12:43:40.520 INFO:tasks.rados.rados.0.smithi138.stdout:1325: expect (ObjNum 578 snap 138 seq_num 578) 2023-06-27T12:43:40.520 INFO:tasks.rados.rados.0.smithi138.stdout:1326: read oid 50 snap 153 2023-06-27T12:43:40.520 INFO:tasks.rados.rados.0.smithi138.stdout:1326: expect deleted 2023-06-27T12:43:40.520 INFO:tasks.rados.rados.0.smithi138.stdout:1327: write oid 30 current snap is 154 2023-06-27T12:43:40.520 INFO:tasks.rados.rados.0.smithi138.stdout:1327: seq_num 640 ranges {756480=733012,2134711=432549} 2023-06-27T12:43:40.526 INFO:tasks.rados.rados.0.smithi138.stdout:1327: writing smithi138138488-30 from 756480 to 1489492 tid 1 2023-06-27T12:43:40.532 INFO:tasks.rados.rados.0.smithi138.stdout:1327: writing smithi138138488-30 from 2134711 to 2567260 tid 2 2023-06-27T12:43:40.533 INFO:tasks.rados.rados.0.smithi138.stdout:1328: snap_remove snap 150 2023-06-27T12:43:41.522 INFO:tasks.rados.rados.0.smithi138.stdout:1328: done (3 left) 2023-06-27T12:43:41.522 INFO:tasks.rados.rados.0.smithi138.stdout:1329: read oid 43 snap -1 2023-06-27T12:43:41.523 INFO:tasks.rados.rados.0.smithi138.stdout:1329: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:43:41.523 INFO:tasks.rados.rados.0.smithi138.stdout:1330: snap_create 2023-06-27T12:43:41.523 INFO:tasks.rados.rados.0.smithi138.stdout:1325: expect (ObjNum 578 snap 138 seq_num 578) 2023-06-27T12:43:41.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:41.758 INFO:tasks.rados.rados.0.smithi138.stdout:1327: finishing write tid 1 to smithi138138488-30 2023-06-27T12:43:41.759 INFO:tasks.rados.rados.0.smithi138.stdout:1327: finishing write tid 2 to smithi138138488-30 2023-06-27T12:43:41.759 INFO:tasks.rados.rados.0.smithi138.stdout:1327: finishing write tid 3 to smithi138138488-30 2023-06-27T12:43:41.759 INFO:tasks.rados.rados.0.smithi138.stdout:1327: finishing write tid 4 to smithi138138488-30 2023-06-27T12:43:41.759 INFO:tasks.rados.rados.0.smithi138.stdout:1327: oid 30 updating version 0 to 228 2023-06-27T12:43:41.759 INFO:tasks.rados.rados.0.smithi138.stdout:1327: oid 30 version 228 is already newer than 228 2023-06-27T12:43:41.759 INFO:tasks.rados.rados.0.smithi138.stdout:1327: oid 30 version 228 is already newer than 228 2023-06-27T12:43:41.759 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 228 (ObjNum 640 snap 154 seq_num 640) dirty exists 2023-06-27T12:43:41.759 INFO:tasks.rados.rados.0.smithi138.stdout:1327: left oid 30 (ObjNum 640 snap 154 seq_num 640) 2023-06-27T12:43:41.760 INFO:tasks.rados.rados.0.smithi138.stdout:1329: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:43:41.766 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:41.761+0000 7f2793c26700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:41.770 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:41.766+0000 7f2793c26700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:41.815 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:41.818 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:41.818 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:42.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~187s 2023-06-27T12:43:42.526 INFO:tasks.rados.rados.0.smithi138.stdout:1325: done (4 left) 2023-06-27T12:43:42.526 INFO:tasks.rados.rados.0.smithi138.stdout:1326: done (3 left) 2023-06-27T12:43:42.526 INFO:tasks.rados.rados.0.smithi138.stdout:1327: done (2 left) 2023-06-27T12:43:42.527 INFO:tasks.rados.rados.0.smithi138.stdout:1329: done (1 left) 2023-06-27T12:43:42.527 INFO:tasks.rados.rados.0.smithi138.stdout:1330: done (0 left) 2023-06-27T12:43:42.527 INFO:tasks.rados.rados.0.smithi138.stdout:1331: read oid 45 snap -1 2023-06-27T12:43:42.527 INFO:tasks.rados.rados.0.smithi138.stdout:1331: expect deleted 2023-06-27T12:43:42.527 INFO:tasks.rados.rados.0.smithi138.stdout:1332: delete oid 15 current snap is 155 2023-06-27T12:43:42.558 INFO:tasks.rados.rados.0.smithi138.stdout:1332: done (1 left) 2023-06-27T12:43:42.559 INFO:tasks.rados.rados.0.smithi138.stdout:1333: delete oid 40 current snap is 155 2023-06-27T12:43:42.566 INFO:tasks.rados.rados.0.smithi138.stdout:1333: done (1 left) 2023-06-27T12:43:42.566 INFO:tasks.rados.rados.0.smithi138.stdout:1334: read oid 10 snap -1 2023-06-27T12:43:42.566 INFO:tasks.rados.rados.0.smithi138.stdout:1334: expect (ObjNum 637 snap 153 seq_num 637) 2023-06-27T12:43:42.567 INFO:tasks.rados.rados.0.smithi138.stdout:1335: read oid 41 snap 153 2023-06-27T12:43:42.567 INFO:tasks.rados.rados.0.smithi138.stdout:1335: expect deleted 2023-06-27T12:43:42.567 INFO:tasks.rados.rados.0.smithi138.stdout:1336: read oid 13 snap 151 2023-06-27T12:43:42.567 INFO:tasks.rados.rados.0.smithi138.stdout:1336: expect deleted 2023-06-27T12:43:42.567 INFO:tasks.rados.rados.0.smithi138.stdout:1337: read oid 11 snap -1 2023-06-27T12:43:42.567 INFO:tasks.rados.rados.0.smithi138.stdout:1337: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:43:42.567 INFO:tasks.rados.rados.0.smithi138.stdout:1338: delete oid 28 current snap is 155 2023-06-27T12:43:42.600 INFO:tasks.rados.rados.0.smithi138.stdout:1337: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:43:42.683 INFO:tasks.rados.rados.0.smithi138.stdout:1334: expect (ObjNum 637 snap 153 seq_num 637) 2023-06-27T12:43:42.846 INFO:tasks.rados.rados.0.smithi138.stdout:1331: done (5 left) 2023-06-27T12:43:42.847 INFO:tasks.rados.rados.0.smithi138.stdout:1334: done (4 left) 2023-06-27T12:43:42.847 INFO:tasks.rados.rados.0.smithi138.stdout:1335: done (3 left) 2023-06-27T12:43:42.847 INFO:tasks.rados.rados.0.smithi138.stdout:1336: done (2 left) 2023-06-27T12:43:42.847 INFO:tasks.rados.rados.0.smithi138.stdout:1337: done (1 left) 2023-06-27T12:43:42.847 INFO:tasks.rados.rados.0.smithi138.stdout:1338: done (0 left) 2023-06-27T12:43:42.847 INFO:tasks.rados.rados.0.smithi138.stdout:1339: delete oid 49 current snap is 155 2023-06-27T12:43:42.849 INFO:tasks.rados.rados.0.smithi138.stdout:1339: done (0 left) 2023-06-27T12:43:42.849 INFO:tasks.rados.rados.0.smithi138.stdout:1340: read oid 48 snap -1 2023-06-27T12:43:42.849 INFO:tasks.rados.rados.0.smithi138.stdout:1340: expect (ObjNum 602 snap 144 seq_num 602) 2023-06-27T12:43:42.850 INFO:tasks.rados.rados.0.smithi138.stdout:1341: write (excl) oid 2 current snap is 155 2023-06-27T12:43:42.850 INFO:tasks.rados.rados.0.smithi138.stdout:1341: seq_num 645 ranges {440515=709257,1910768=583545} 2023-06-27T12:43:42.856 INFO:tasks.rados.rados.0.smithi138.stdout:1341: writing smithi138138488-2 from 440515 to 1149772 tid 1 2023-06-27T12:43:42.864 INFO:tasks.rados.rados.0.smithi138.stdout:1341: writing smithi138138488-2 from 1910768 to 2494313 tid 2 2023-06-27T12:43:42.864 INFO:tasks.rados.rados.0.smithi138.stdout:1342: snap_create 2023-06-27T12:43:42.864 INFO:tasks.rados.rados.0.smithi138.stdout:1340: expect (ObjNum 602 snap 144 seq_num 602) 2023-06-27T12:43:42.997 INFO:tasks.rados.rados.0.smithi138.stdout:1341: finishing write tid 1 to smithi138138488-2 2023-06-27T12:43:42.998 INFO:tasks.rados.rados.0.smithi138.stdout:1341: finishing write tid 2 to smithi138138488-2 2023-06-27T12:43:42.998 INFO:tasks.rados.rados.0.smithi138.stdout:1341: finishing write tid 3 to smithi138138488-2 2023-06-27T12:43:42.998 INFO:tasks.rados.rados.0.smithi138.stdout:1341: finishing write tid 4 to smithi138138488-2 2023-06-27T12:43:42.998 INFO:tasks.rados.rados.0.smithi138.stdout:1341: oid 2 updating version 0 to 414 2023-06-27T12:43:42.998 INFO:tasks.rados.rados.0.smithi138.stdout:1341: oid 2 updating version 414 to 417 2023-06-27T12:43:42.998 INFO:tasks.rados.rados.0.smithi138.stdout:1341: oid 2 version 417 is already newer than 417 2023-06-27T12:43:42.999 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 417 (ObjNum 645 snap 155 seq_num 645) dirty exists 2023-06-27T12:43:42.999 INFO:tasks.rados.rados.0.smithi138.stdout:1341: left oid 2 (ObjNum 645 snap 155 seq_num 645) 2023-06-27T12:43:43.529 INFO:tasks.rados.rados.0.smithi138.stdout:1340: done (2 left) 2023-06-27T12:43:43.530 INFO:tasks.rados.rados.0.smithi138.stdout:1341: done (1 left) 2023-06-27T12:43:43.530 INFO:tasks.rados.rados.0.smithi138.stdout:1342: done (0 left) 2023-06-27T12:43:43.530 INFO:tasks.rados.rados.0.smithi138.stdout:1343: write oid 23 current snap is 156 2023-06-27T12:43:43.530 INFO:tasks.rados.rados.0.smithi138.stdout:1343: seq_num 646 ranges {457322=784506,1969250=443556} 2023-06-27T12:43:43.536 INFO:tasks.rados.rados.0.smithi138.stdout:1343: writing smithi138138488-23 from 457322 to 1241828 tid 1 2023-06-27T12:43:43.543 INFO:tasks.rados.rados.0.smithi138.stdout:1343: writing smithi138138488-23 from 1969250 to 2412806 tid 2 2023-06-27T12:43:43.543 INFO:tasks.rados.rados.0.smithi138.stdout:1344: snap_remove snap 154 2023-06-27T12:43:43.820 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:43.949 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:43.944+0000 7f1e58d2a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:43.948+0000 7f1e58d2a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:43.998 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:44.001 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:44.001 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:44.533 INFO:tasks.rados.rados.0.smithi138.stdout:1344: done (1 left) 2023-06-27T12:43:44.534 INFO:tasks.rados.rados.0.smithi138.stdout:1345: delete oid 21 current snap is 156 2023-06-27T12:43:44.534 INFO:tasks.rados.rados.0.smithi138.stdout:1343: finishing write tid 1 to smithi138138488-23 2023-06-27T12:43:44.534 INFO:tasks.rados.rados.0.smithi138.stdout:1343: finishing write tid 2 to smithi138138488-23 2023-06-27T12:43:44.534 INFO:tasks.rados.rados.0.smithi138.stdout:1343: finishing write tid 3 to smithi138138488-23 2023-06-27T12:43:44.534 INFO:tasks.rados.rados.0.smithi138.stdout:1343: finishing write tid 4 to smithi138138488-23 2023-06-27T12:43:44.534 INFO:tasks.rados.rados.0.smithi138.stdout:1343: oid 23 updating version 0 to 179 2023-06-27T12:43:44.535 INFO:tasks.rados.rados.0.smithi138.stdout:1343: oid 23 version 179 is already newer than 179 2023-06-27T12:43:44.535 INFO:tasks.rados.rados.0.smithi138.stdout:1343: oid 23 version 179 is already newer than 179 2023-06-27T12:43:44.535 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 179 (ObjNum 646 snap 156 seq_num 646) dirty exists 2023-06-27T12:43:44.535 INFO:tasks.rados.rados.0.smithi138.stdout:1343: left oid 23 (ObjNum 646 snap 156 seq_num 646) 2023-06-27T12:43:44.558 INFO:tasks.rados.rados.0.smithi138.stdout:1343: done (1 left) 2023-06-27T12:43:44.559 INFO:tasks.rados.rados.0.smithi138.stdout:1345: done (0 left) 2023-06-27T12:43:44.559 INFO:tasks.rados.rados.0.smithi138.stdout:1346: write oid 46 current snap is 156 2023-06-27T12:43:44.559 INFO:tasks.rados.rados.0.smithi138.stdout:1346: seq_num 648 ranges {490936=651357,1686214=447225,2861864=29072} 2023-06-27T12:43:44.564 INFO:tasks.rados.rados.0.smithi138.stdout:1346: writing smithi138138488-46 from 490936 to 1142293 tid 1 2023-06-27T12:43:44.570 INFO:tasks.rados.rados.0.smithi138.stdout:1346: writing smithi138138488-46 from 1686214 to 2133439 tid 2 2023-06-27T12:43:44.574 INFO:tasks.rados.rados.0.smithi138.stdout:1346: writing smithi138138488-46 from 2861864 to 2890936 tid 3 2023-06-27T12:43:44.575 INFO:tasks.rados.rados.0.smithi138.stdout:1347: write oid 25 current snap is 156 2023-06-27T12:43:44.575 INFO:tasks.rados.rados.0.smithi138.stdout:1347: seq_num 649 ranges {507743=726606,1744696=590883,2772934=134809} 2023-06-27T12:43:44.581 INFO:tasks.rados.rados.0.smithi138.stdout:1347: writing smithi138138488-25 from 507743 to 1234349 tid 1 2023-06-27T12:43:44.588 INFO:tasks.rados.rados.0.smithi138.stdout:1347: writing smithi138138488-25 from 1744696 to 2335579 tid 2 2023-06-27T12:43:44.592 INFO:tasks.rados.rados.0.smithi138.stdout:1347: writing smithi138138488-25 from 2772934 to 2907743 tid 3 2023-06-27T12:43:44.593 INFO:tasks.rados.rados.0.smithi138.stdout:1348: write (excl) oid 15 current snap is 156 2023-06-27T12:43:44.593 INFO:tasks.rados.rados.0.smithi138.stdout:1348: seq_num 650 ranges {524550=401855,1403178=450894,2516710=407840} 2023-06-27T12:43:44.599 INFO:tasks.rados.rados.0.smithi138.stdout:1348: writing smithi138138488-15 from 524550 to 926405 tid 1 2023-06-27T12:43:44.604 INFO:tasks.rados.rados.0.smithi138.stdout:1348: writing smithi138138488-15 from 1403178 to 1854072 tid 2 2023-06-27T12:43:44.610 INFO:tasks.rados.rados.0.smithi138.stdout:1348: writing smithi138138488-15 from 2516710 to 2924550 tid 3 2023-06-27T12:43:44.610 INFO:tasks.rados.rados.0.smithi138.stdout:1349: snap_remove snap 149 2023-06-27T12:43:44.618 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:44.618 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:44.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:44.758 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:44.751+0000 7f2cdac0f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:44.761 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:44.757+0000 7f2cdac0f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:44.954 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:44.956 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:44.968 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2023-06-27T12:43:44.528586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":89147958,"num_objects":334,"num_object_clones":287,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1951,"num_read_kb":1416839,"num_write":1754,"num_write_kb":509664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3199,"ondisk_log_size":3199,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":984496,"kb_used_data":848528,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376502864,"statfs":{"total":386547056640,"available":385538932736,"internally_reserved":0,"allocated":868892672,"data_stored":989064962,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-605019,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":25527,"num_write":12,"num_write_kb":3603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015827"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":787,"reported_epoch":330,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:43.532317+0000","last_change":"2023-06-27T12:43:41.544796+0000","last_active":"2023-06-27T12:43:43.532317+0000","last_peered":"2023-06-27T12:43:43.532317+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:43.532317+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"316'188","reported_seq":1029,"reported_epoch":330,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:43.555714+0000","last_change":"2023-06-27T12:43:41.540174+0000","last_active":"2023-06-27T12:43:43.555714+0000","last_peered":"2023-06-27T12:43:43.555714+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:43.555714+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":177491,"num_write":137,"num_write_kb":40636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":24}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"325'268","reported_seq":1020,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:42.588476+0000","last_change":"2023-06-27T12:43:41.993168+0000","last_active":"2023-06-27T12:43:42.588476+0000","last_peered":"2023-06-27T12:43:42.588476+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:42.588476+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8202146,"num_objects":30,"num_object_clones":25,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":215,"num_read_kb":137557,"num_write":169,"num_write_kb":48913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"321'151","reported_seq":967,"reported_epoch":330,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:43.540137+0000","last_change":"2023-06-27T12:43:41.571635+0000","last_active":"2023-06-27T12:43:43.540137+0000","last_peered":"2023-06-27T12:43:43.540137+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:43.540137+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512927,"num_objects":25,"num_object_clones":21,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":152,"num_read_kb":102691,"num_write":110,"num_write_kb":31279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":25}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"305'91","reported_seq":514,"reported_epoch":330,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:43.539917+0000","last_change":"2023-06-27T12:43:41.539860+0000","last_active":"2023-06-27T12:43:43.539917+0000","last_peered":"2023-06-27T12:43:43.539917+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:43.539917+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3848902,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":88,"num_read_kb":50182,"num_write":49,"num_write_kb":15136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"323'389","reported_seq":1057,"reported_epoch":330,"state":"active+clean","last_fresh":"2023-06-27T12:43:43.537649+0000","last_change":"2023-06-27T12:43:41.568524+0000","last_active":"2023-06-27T12:43:43.537649+0000","last_peered":"2023-06-27T12:43:43.537649+0000","last_clean":"2023-06-27T12:43:43.537649+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:43.537649+0000","last_undegraded":"2023-06-27T12:43:43.537649+0000","last_fullsized":"2023-06-27T12:43:43.537649+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5059171,"num_objects":44,"num_object_clones":38,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":306,"num_read_kb":193213,"num_write":232,"num_write_kb":66787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"325'183","reported_seq":867,"reported_epoch":330,"state":"active+clean","last_fresh":"2023-06-27T12:43:43.536802+0000","last_change":"2023-06-27T12:43:41.546692+0000","last_active":"2023-06-27T12:43:43.536802+0000","last_peered":"2023-06-27T12:43:43.536802+0000","last_clean":"2023-06-27T12:43:43.536802+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:43.536802+0000","last_undegraded":"2023-06-27T12:43:43.536802+0000","last_fullsized":"2023-06-27T12:43:43.536802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521347,"num_objects":22,"num_object_clones":19,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":127,"num_read_kb":85547,"num_write":112,"num_write_kb":32865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"320'69","reported_seq":821,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:42.553081+0000","last_change":"2023-06-27T12:43:41.988950+0000","last_active":"2023-06-27T12:43:42.553081+0000","last_peered":"2023-06-27T12:43:42.553081+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:42.553081+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":64,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":552,"reported_epoch":330,"state":"active+undersized","last_fresh":"2023-06-27T12:43:43.547811+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:43.547811+0000","last_peered":"2023-06-27T12:43:43.547811+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:43.547811+0000","last_undegraded":"2023-06-27T12:43:43.547811+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":734,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-06-27T12:43:42.550817+0000","last_change":"2023-06-27T12:43:41.993592+0000","last_active":"2023-06-27T12:43:42.550817+0000","last_peered":"2023-06-27T12:43:42.550817+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:42.550817+0000","last_undegraded":"2023-06-27T12:43:42.550817+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":535,"reported_epoch":330,"state":"active+clean","last_fresh":"2023-06-27T12:43:43.539212+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:43.539212+0000","last_peered":"2023-06-27T12:43:43.539212+0000","last_clean":"2023-06-27T12:43:43.539212+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:43.539212+0000","last_undegraded":"2023-06-27T12:43:43.539212+0000","last_fullsized":"2023-06-27T12:43:43.539212+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"330'179","reported_seq":947,"reported_epoch":330,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:43.574966+0000","last_change":"2023-06-27T12:43:41.541316+0000","last_active":"2023-06-27T12:43:43.574966+0000","last_peered":"2023-06-27T12:43:43.574966+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:43.574966+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5664190,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":75,"num_read_kb":70436,"num_write":114,"num_write_kb":31932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":518,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-06-27T12:43:42.558427+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:42.558427+0000","last_peered":"2023-06-27T12:43:42.558427+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:42.558427+0000","last_undegraded":"2023-06-27T12:43:42.558427+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"329'417","reported_seq":613,"reported_epoch":330,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:43.550276+0000","last_change":"2023-06-27T12:43:41.540928+0000","last_active":"2023-06-27T12:43:43.550276+0000","last_peered":"2023-06-27T12:43:43.550276+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:43.550276+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4370051,"num_objects":22,"num_object_clones":20,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":59,"num_read_kb":33574,"num_write":112,"num_write_kb":34668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":515,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-06-27T12:43:42.539544+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:42.539544+0000","last_peered":"2023-06-27T12:43:42.539544+0000","last_clean":"2023-06-27T12:43:42.539544+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:42.539544+0000","last_undegraded":"2023-06-27T12:43:42.539544+0000","last_fullsized":"2023-06-27T12:43:42.539544+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":287,"reported_epoch":330,"state":"active+undersized","last_fresh":"2023-06-27T12:43:43.539223+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:43.539223+0000","last_peered":"2023-06-27T12:43:43.539223+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:43.539223+0000","last_undegraded":"2023-06-27T12:43:43.539223+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"320'370","reported_seq":1156,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:42.551430+0000","last_change":"2023-06-27T12:43:41.989301+0000","last_active":"2023-06-27T12:43:42.551430+0000","last_peered":"2023-06-27T12:43:42.551430+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:42.551430+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521347,"num_objects":49,"num_object_clones":43,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":300,"num_read_kb":178400,"num_write":269,"num_write_kb":78571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":49}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":535,"reported_epoch":330,"state":"active+clean","last_fresh":"2023-06-27T12:43:43.534312+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:43.534312+0000","last_peered":"2023-06-27T12:43:43.534312+0000","last_clean":"2023-06-27T12:43:43.534312+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:43.534312+0000","last_undegraded":"2023-06-27T12:43:43.534312+0000","last_fullsized":"2023-06-27T12:43:43.534312+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"327'228","reported_seq":545,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:42.566780+0000","last_change":"2023-06-27T12:43:41.996843+0000","last_active":"2023-06-27T12:43:42.566780+0000","last_peered":"2023-06-27T12:43:42.566780+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:42.566780+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7782070,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":53235,"num_write":87,"num_write_kb":24280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":295,"reported_epoch":330,"state":"active+undersized","last_fresh":"2023-06-27T12:43:43.548520+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:43.548520+0000","last_peered":"2023-06-27T12:43:43.548520+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:43.548520+0000","last_undegraded":"2023-06-27T12:43:43.548520+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"325'197","reported_seq":985,"reported_epoch":330,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:43.549601+0000","last_change":"2023-06-27T12:43:41.543034+0000","last_active":"2023-06-27T12:43:43.549601+0000","last_peered":"2023-06-27T12:43:43.549601+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:43.549601+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7361862,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":105,"num_read_kb":112952,"num_write":131,"num_write_kb":37411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"310'115","reported_seq":505,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:42.553058+0000","last_change":"2023-06-27T12:43:41.988704+0000","last_active":"2023-06-27T12:43:42.553058+0000","last_peered":"2023-06-27T12:43:42.553058+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:42.553058+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933102,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":53577,"num_write":43,"num_write_kb":12009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":552,"reported_epoch":330,"state":"active+undersized","last_fresh":"2023-06-27T12:43:43.552939+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:43.552939+0000","last_peered":"2023-06-27T12:43:43.552939+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:43.552939+0000","last_undegraded":"2023-06-27T12:43:43.552939+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":515,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-06-27T12:43:42.557178+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:42.557178+0000","last_peered":"2023-06-27T12:43:42.557178+0000","last_clean":"2023-06-27T12:43:42.557178+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:42.557178+0000","last_undegraded":"2023-06-27T12:43:42.557178+0000","last_fullsized":"2023-06-27T12:43:42.557178+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"305'220","reported_seq":992,"reported_epoch":330,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:43.554593+0000","last_change":"2023-06-27T12:43:41.536020+0000","last_active":"2023-06-27T12:43:43.554593+0000","last_peered":"2023-06-27T12:43:43.554593+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:43.554593+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832260,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":116,"num_read_kb":120851,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":89147958,"num_objects":334,"num_object_clones":287,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1951,"num_read_kb":1416839,"num_write":1754,"num_write_kb":509664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3102,"ondisk_log_size":3102,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673310,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":253224,"kb_used_data":218208,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94118616,"statfs":{"total":96636764160,"available":96377462784,"internally_reserved":0,"allocated":223444992,"data_stored":253834750,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":262732,"kb_used_data":227716,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94109108,"statfs":{"total":96636764160,"available":96367726592,"internally_reserved":0,"allocated":233181184,"data_stored":263909010,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673310,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":287260,"kb_used_data":252244,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94084580,"statfs":{"total":96636764160,"available":96342609920,"internally_reserved":0,"allocated":258297856,"data_stored":294281650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:44.970 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.e 3.d 3.c 3.b 3.3 3.0 3.1 3.7 2023-06-27T12:43:45.105 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:45.089+0000 7fb25ceba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:45.108 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:45.104+0000 7fb25ceba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:45.301 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.c] on osd.0 to force-recovery; instructing pg(s) [3.1,3.7,3.d] on osd.1 to force-recovery; instructing pg(s) [3.0,3.3,3.e] on osd.2 to force-recovery; 2023-06-27T12:43:45.302 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:45.450 INFO:tasks.rados.rados.0.smithi138.stdout:1349: done (3 left) 2023-06-27T12:43:45.450 INFO:tasks.rados.rados.0.smithi138.stdout:1350: snap_remove snap 155 2023-06-27T12:43:46.002 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:46.129 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:46.123+0000 7fd1cedab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:46.132 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:46.128+0000 7fd1cedab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:46.182 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:46.185 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:46.186 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:46.455 INFO:tasks.rados.rados.0.smithi138.stdout:1350: done (3 left) 2023-06-27T12:43:46.455 INFO:tasks.rados.rados.0.smithi138.stdout:1351: snap_remove snap 151 2023-06-27T12:43:47.458 INFO:tasks.rados.rados.0.smithi138.stdout:1351: done (3 left) 2023-06-27T12:43:47.458 INFO:tasks.rados.rados.0.smithi138.stdout:1352: delete oid 24 current snap is 156 2023-06-27T12:43:47.458 INFO:tasks.rados.rados.0.smithi138.stdout:1346: finishing write tid 1 to smithi138138488-46 2023-06-27T12:43:47.459 INFO:tasks.rados.rados.0.smithi138.stdout:1347: finishing write tid 1 to smithi138138488-25 2023-06-27T12:43:47.459 INFO:tasks.rados.rados.0.smithi138.stdout:1346: finishing write tid 2 to smithi138138488-46 2023-06-27T12:43:47.459 INFO:tasks.rados.rados.0.smithi138.stdout:1346: finishing write tid 3 to smithi138138488-46 2023-06-27T12:43:47.459 INFO:tasks.rados.rados.0.smithi138.stdout:1346: finishing write tid 4 to smithi138138488-46 2023-06-27T12:43:47.459 INFO:tasks.rados.rados.0.smithi138.stdout:1346: finishing write tid 5 to smithi138138488-46 2023-06-27T12:43:47.459 INFO:tasks.rados.rados.0.smithi138.stdout:1346: oid 46 updating version 0 to 185 2023-06-27T12:43:47.459 INFO:tasks.rados.rados.0.smithi138.stdout:1346: oid 46 version 185 is already newer than 185 2023-06-27T12:43:47.459 INFO:tasks.rados.rados.0.smithi138.stdout:1346: oid 46 version 185 is already newer than 185 2023-06-27T12:43:47.460 INFO:tasks.rados.rados.0.smithi138.stdout:1346: oid 46 version 185 is already newer than 185 2023-06-27T12:43:47.460 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 185 (ObjNum 648 snap 156 seq_num 648) dirty exists 2023-06-27T12:43:47.460 INFO:tasks.rados.rados.0.smithi138.stdout:1346: left oid 46 (ObjNum 648 snap 156 seq_num 648) 2023-06-27T12:43:47.460 INFO:tasks.rados.rados.0.smithi138.stdout:1347: finishing write tid 2 to smithi138138488-25 2023-06-27T12:43:47.460 INFO:tasks.rados.rados.0.smithi138.stdout:1347: finishing write tid 3 to smithi138138488-25 2023-06-27T12:43:47.460 INFO:tasks.rados.rados.0.smithi138.stdout:1347: finishing write tid 4 to smithi138138488-25 2023-06-27T12:43:47.460 INFO:tasks.rados.rados.0.smithi138.stdout:1347: finishing write tid 5 to smithi138138488-25 2023-06-27T12:43:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:1347: oid 25 updating version 0 to 121 2023-06-27T12:43:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:1347: oid 25 updating version 121 to 122 2023-06-27T12:43:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:1347: oid 25 version 122 is already newer than 122 2023-06-27T12:43:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:1347: oid 25 version 122 is already newer than 122 2023-06-27T12:43:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 122 (ObjNum 649 snap 156 seq_num 649) dirty exists 2023-06-27T12:43:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:1347: left oid 25 (ObjNum 649 snap 156 seq_num 649) 2023-06-27T12:43:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:1348: finishing write tid 1 to smithi138138488-15 2023-06-27T12:43:47.461 INFO:tasks.rados.rados.0.smithi138.stdout:1348: finishing write tid 2 to smithi138138488-15 2023-06-27T12:43:47.462 INFO:tasks.rados.rados.0.smithi138.stdout:1348: finishing write tid 3 to smithi138138488-15 2023-06-27T12:43:47.462 INFO:tasks.rados.rados.0.smithi138.stdout:1348: finishing write tid 4 to smithi138138488-15 2023-06-27T12:43:47.462 INFO:tasks.rados.rados.0.smithi138.stdout:1348: finishing write tid 5 to smithi138138488-15 2023-06-27T12:43:47.462 INFO:tasks.rados.rados.0.smithi138.stdout:1348: oid 15 updating version 0 to 100 2023-06-27T12:43:47.462 INFO:tasks.rados.rados.0.smithi138.stdout:1348: oid 15 version 100 is already newer than 97 2023-06-27T12:43:47.462 INFO:tasks.rados.rados.0.smithi138.stdout:1348: oid 15 version 100 is already newer than 100 2023-06-27T12:43:47.462 INFO:tasks.rados.rados.0.smithi138.stdout:1348: oid 15 version 100 is already newer than 100 2023-06-27T12:43:47.462 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 100 (ObjNum 650 snap 156 seq_num 650) dirty exists 2023-06-27T12:43:47.463 INFO:tasks.rados.rados.0.smithi138.stdout:1348: left oid 15 (ObjNum 650 snap 156 seq_num 650) 2023-06-27T12:43:47.593 INFO:tasks.rados.rados.0.smithi138.stdout:1346: done (3 left) 2023-06-27T12:43:47.593 INFO:tasks.rados.rados.0.smithi138.stdout:1347: done (2 left) 2023-06-27T12:43:47.593 INFO:tasks.rados.rados.0.smithi138.stdout:1348: done (1 left) 2023-06-27T12:43:47.594 INFO:tasks.rados.rados.0.smithi138.stdout:1352: done (0 left) 2023-06-27T12:43:47.594 INFO:tasks.rados.rados.0.smithi138.stdout:1353: read oid 29 snap 153 2023-06-27T12:43:47.594 INFO:tasks.rados.rados.0.smithi138.stdout:1353: expect (ObjNum 610 snap 145 seq_num 610) 2023-06-27T12:43:47.594 INFO:tasks.rados.rados.0.smithi138.stdout:1354: read oid 27 snap -1 2023-06-27T12:43:47.594 INFO:tasks.rados.rados.0.smithi138.stdout:1354: expect (ObjNum 623 snap 147 seq_num 623) 2023-06-27T12:43:47.594 INFO:tasks.rados.rados.0.smithi138.stdout:1355: read oid 12 snap -1 2023-06-27T12:43:47.595 INFO:tasks.rados.rados.0.smithi138.stdout:1355: expect (ObjNum 551 snap 132 seq_num 551) 2023-06-27T12:43:47.595 INFO:tasks.rados.rados.0.smithi138.stdout:1356: read oid 8 snap -1 2023-06-27T12:43:47.595 INFO:tasks.rados.rados.0.smithi138.stdout:1356: expect (ObjNum 586 snap 140 seq_num 586) 2023-06-27T12:43:47.595 INFO:tasks.rados.rados.0.smithi138.stdout:1357: delete oid 23 current snap is 156 2023-06-27T12:43:47.606 INFO:tasks.rados.rados.0.smithi138.stdout:1353: expect (ObjNum 610 snap 145 seq_num 610) 2023-06-27T12:43:47.725 INFO:tasks.rados.rados.0.smithi138.stdout:1356: expect (ObjNum 586 snap 140 seq_num 586) 2023-06-27T12:43:47.909 INFO:tasks.rados.rados.0.smithi138.stdout:1355: expect (ObjNum 551 snap 132 seq_num 551) 2023-06-27T12:43:47.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~193s 2023-06-27T12:43:48.127 INFO:tasks.rados.rados.0.smithi138.stdout:1354: expect (ObjNum 623 snap 147 seq_num 623) 2023-06-27T12:43:48.187 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:48.278 INFO:tasks.rados.rados.0.smithi138.stdout:1353: done (4 left) 2023-06-27T12:43:48.278 INFO:tasks.rados.rados.0.smithi138.stdout:1354: done (3 left) 2023-06-27T12:43:48.278 INFO:tasks.rados.rados.0.smithi138.stdout:1355: done (2 left) 2023-06-27T12:43:48.279 INFO:tasks.rados.rados.0.smithi138.stdout:1356: done (1 left) 2023-06-27T12:43:48.279 INFO:tasks.rados.rados.0.smithi138.stdout:1357: done (0 left) 2023-06-27T12:43:48.279 INFO:tasks.rados.rados.0.smithi138.stdout:1358: write oid 30 current snap is 156 2023-06-27T12:43:48.279 INFO:tasks.rados.rados.0.smithi138.stdout:1358: seq_num 653 ranges {574971=627602,1862271=598221} 2023-06-27T12:43:48.287 INFO:tasks.rados.rados.0.smithi138.stdout:1358: writing smithi138138488-30 from 574971 to 1202573 tid 1 2023-06-27T12:43:48.298 INFO:tasks.rados.rados.0.smithi138.stdout:1358: writing smithi138138488-30 from 1862271 to 2460492 tid 2 2023-06-27T12:43:48.298 INFO:tasks.rados.rados.0.smithi138.stdout:1359: snap_create 2023-06-27T12:43:48.305 INFO:tasks.rados.rados.0.smithi138.stdout:1358: finishing write tid 1 to smithi138138488-30 2023-06-27T12:43:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:48.317+0000 7f2eb1579700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:48.325 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:48.322+0000 7f2eb1579700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:48.327 INFO:tasks.rados.rados.0.smithi138.stdout:1358: finishing write tid 2 to smithi138138488-30 2023-06-27T12:43:48.329 INFO:tasks.rados.rados.0.smithi138.stdout:1358: finishing write tid 3 to smithi138138488-30 2023-06-27T12:43:48.334 INFO:tasks.rados.rados.0.smithi138.stdout:1358: finishing write tid 4 to smithi138138488-30 2023-06-27T12:43:48.334 INFO:tasks.rados.rados.0.smithi138.stdout:1358: oid 30 updating version 0 to 237 2023-06-27T12:43:48.334 INFO:tasks.rados.rados.0.smithi138.stdout:1358: oid 30 version 237 is already newer than 237 2023-06-27T12:43:48.334 INFO:tasks.rados.rados.0.smithi138.stdout:1358: oid 30 version 237 is already newer than 235 2023-06-27T12:43:48.334 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 237 (ObjNum 653 snap 156 seq_num 653) dirty exists 2023-06-27T12:43:48.334 INFO:tasks.rados.rados.0.smithi138.stdout:1358: left oid 30 (ObjNum 653 snap 156 seq_num 653) 2023-06-27T12:43:48.372 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:48.375 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:48.375 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:48.545 INFO:tasks.rados.rados.0.smithi138.stdout:1358: done (1 left) 2023-06-27T12:43:48.545 INFO:tasks.rados.rados.0.smithi138.stdout:1359: done (0 left) 2023-06-27T12:43:48.545 INFO:tasks.rados.rados.0.smithi138.stdout:1360: delete oid 16 current snap is 157 2023-06-27T12:43:48.562 INFO:tasks.rados.rados.0.smithi138.stdout:1360: done (0 left) 2023-06-27T12:43:48.563 INFO:tasks.rados.rados.0.smithi138.stdout:1361: snap_create 2023-06-27T12:43:49.547 INFO:tasks.rados.rados.0.smithi138.stdout:1361: done (0 left) 2023-06-27T12:43:49.547 INFO:tasks.rados.rados.0.smithi138.stdout:1362: write (excl) oid 50 current snap is 158 2023-06-27T12:43:49.547 INFO:tasks.rados.rados.0.smithi138.stdout:1362: seq_num 655 ranges {608585=494453,1579235=601890,2821119=187466} 2023-06-27T12:43:49.554 INFO:tasks.rados.rados.0.smithi138.stdout:1362: writing smithi138138488-50 from 608585 to 1103038 tid 1 2023-06-27T12:43:49.560 INFO:tasks.rados.rados.0.smithi138.stdout:1362: writing smithi138138488-50 from 1579235 to 2181125 tid 2 2023-06-27T12:43:49.565 INFO:tasks.rados.rados.0.smithi138.stdout:1362: writing smithi138138488-50 from 2821119 to 3008585 tid 3 2023-06-27T12:43:49.565 INFO:tasks.rados.rados.0.smithi138.stdout:1363: read oid 37 snap 156 2023-06-27T12:43:49.565 INFO:tasks.rados.rados.0.smithi138.stdout:1363: expect deleted 2023-06-27T12:43:49.565 INFO:tasks.rados.rados.0.smithi138.stdout:1364: read oid 36 snap 153 2023-06-27T12:43:49.565 INFO:tasks.rados.rados.0.smithi138.stdout:1364: expect (ObjNum 639 snap 153 seq_num 639) 2023-06-27T12:43:49.566 INFO:tasks.rados.rados.0.smithi138.stdout:1365: read oid 2 snap 156 2023-06-27T12:43:49.566 INFO:tasks.rados.rados.0.smithi138.stdout:1365: expect (ObjNum 645 snap 155 seq_num 645) 2023-06-27T12:43:49.566 INFO:tasks.rados.rados.0.smithi138.stdout:1366: read oid 15 snap -1 2023-06-27T12:43:49.566 INFO:tasks.rados.rados.0.smithi138.stdout:1366: expect (ObjNum 650 snap 156 seq_num 650) 2023-06-27T12:43:49.566 INFO:tasks.rados.rados.0.smithi138.stdout:1367: read oid 9 snap -1 2023-06-27T12:43:49.566 INFO:tasks.rados.rados.0.smithi138.stdout:1367: expect (ObjNum 594 snap 142 seq_num 594) 2023-06-27T12:43:49.566 INFO:tasks.rados.rados.0.smithi138.stdout:1368: snap_create 2023-06-27T12:43:49.614 INFO:tasks.rados.rados.0.smithi138.stdout:1366: expect (ObjNum 650 snap 156 seq_num 650) 2023-06-27T12:43:49.733 INFO:tasks.rados.rados.0.smithi138.stdout:1365: expect (ObjNum 645 snap 155 seq_num 645) 2023-06-27T12:43:49.882 INFO:tasks.rados.rados.0.smithi138.stdout:1364: expect (ObjNum 639 snap 153 seq_num 639) 2023-06-27T12:43:49.988 INFO:tasks.rados.rados.0.smithi138.stdout:1362: finishing write tid 1 to smithi138138488-50 2023-06-27T12:43:49.988 INFO:tasks.rados.rados.0.smithi138.stdout:1367: expect (ObjNum 594 snap 142 seq_num 594) 2023-06-27T12:43:50.249 INFO:tasks.rados.rados.0.smithi138.stdout:1362: finishing write tid 2 to smithi138138488-50 2023-06-27T12:43:50.249 INFO:tasks.rados.rados.0.smithi138.stdout:1362: finishing write tid 3 to smithi138138488-50 2023-06-27T12:43:50.250 INFO:tasks.rados.rados.0.smithi138.stdout:1362: finishing write tid 4 to smithi138138488-50 2023-06-27T12:43:50.250 INFO:tasks.rados.rados.0.smithi138.stdout:1362: finishing write tid 5 to smithi138138488-50 2023-06-27T12:43:50.250 INFO:tasks.rados.rados.0.smithi138.stdout:1362: oid 50 updating version 0 to 79 2023-06-27T12:43:50.250 INFO:tasks.rados.rados.0.smithi138.stdout:1362: oid 50 version 79 is already newer than 79 2023-06-27T12:43:50.251 INFO:tasks.rados.rados.0.smithi138.stdout:1362: oid 50 version 79 is already newer than 76 2023-06-27T12:43:50.251 INFO:tasks.rados.rados.0.smithi138.stdout:1362: oid 50 version 79 is already newer than 79 2023-06-27T12:43:50.251 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 79 (ObjNum 655 snap 158 seq_num 655) dirty exists 2023-06-27T12:43:50.252 INFO:tasks.rados.rados.0.smithi138.stdout:1362: left oid 50 (ObjNum 655 snap 158 seq_num 655) 2023-06-27T12:43:50.313 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:50.313 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:50.314 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:50.376 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:50.446 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:50.439+0000 7fc1fb4fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:50.451 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:50.447+0000 7fc1fb4fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:50.507 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:50.501+0000 7ff41bc1e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:50.510 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:50.506+0000 7ff41bc1e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:50.559 INFO:tasks.rados.rados.0.smithi138.stdout:1362: done (6 left) 2023-06-27T12:43:50.559 INFO:tasks.rados.rados.0.smithi138.stdout:1363: done (5 left) 2023-06-27T12:43:50.560 INFO:tasks.rados.rados.0.smithi138.stdout:1364: done (4 left) 2023-06-27T12:43:50.560 INFO:tasks.rados.rados.0.smithi138.stdout:1365: done (3 left) 2023-06-27T12:43:50.560 INFO:tasks.rados.rados.0.smithi138.stdout:1366: done (2 left) 2023-06-27T12:43:50.560 INFO:tasks.rados.rados.0.smithi138.stdout:1367: done (1 left) 2023-06-27T12:43:50.560 INFO:tasks.rados.rados.0.smithi138.stdout:1368: done (0 left) 2023-06-27T12:43:50.560 INFO:tasks.rados.rados.0.smithi138.stdout:1369: snap_remove snap 158 2023-06-27T12:43:50.563 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:50.566 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:50.566 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:50.678 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:50.691 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2023-06-27T12:43:50.551649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88929500,"num_objects":336,"num_object_clones":289,"num_object_copies":1008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1982,"num_read_kb":1442293,"num_write":1773,"num_write_kb":514618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3252,"ondisk_log_size":3252,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":978536,"kb_used_data":841064,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376508824,"statfs":{"total":386547056640,"available":385545035776,"internally_reserved":0,"allocated":861249536,"data_stored":975889046,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-218458,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":25454,"num_write":19,"num_write_kb":4954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.102070"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":801,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.555020+0000","last_change":"2023-06-27T12:43:47.488350+0000","last_active":"2023-06-27T12:43:48.555020+0000","last_peered":"2023-06-27T12:43:48.555020+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:48.555020+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"316'188","reported_seq":1045,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.556593+0000","last_change":"2023-06-27T12:43:47.619843+0000","last_active":"2023-06-27T12:43:48.556593+0000","last_peered":"2023-06-27T12:43:48.556593+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:48.556593+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":182317,"num_write":137,"num_write_kb":40636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":24}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"325'268","reported_seq":1037,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.574095+0000","last_change":"2023-06-27T12:43:47.503294+0000","last_active":"2023-06-27T12:43:48.574095+0000","last_peered":"2023-06-27T12:43:48.574095+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:48.574095+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8202146,"num_objects":30,"num_object_clones":25,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":222,"num_read_kb":146611,"num_write":169,"num_write_kb":48913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"321'151","reported_seq":982,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.552912+0000","last_change":"2023-06-27T12:43:47.523529+0000","last_active":"2023-06-27T12:43:48.552912+0000","last_peered":"2023-06-27T12:43:48.552912+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:48.552912+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512927,"num_objects":25,"num_object_clones":21,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":152,"num_read_kb":102691,"num_write":110,"num_write_kb":31279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":25}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"331'100","reported_seq":536,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.555248+0000","last_change":"2023-06-27T12:43:47.488323+0000","last_active":"2023-06-27T12:43:48.555248+0000","last_peered":"2023-06-27T12:43:48.555248+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:48.555248+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6773452,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":97,"num_read_kb":58366,"num_write":54,"num_write_kb":16369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"334'408","reported_seq":1073,"reported_epoch":335,"state":"active+clean","last_fresh":"2023-06-27T12:43:48.552905+0000","last_change":"2023-06-27T12:43:47.554763+0000","last_active":"2023-06-27T12:43:48.552905+0000","last_peered":"2023-06-27T12:43:48.552905+0000","last_clean":"2023-06-27T12:43:48.552905+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:48.552905+0000","last_undegraded":"2023-06-27T12:43:48.552905+0000","last_fullsized":"2023-06-27T12:43:48.552905+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2437147,"num_objects":45,"num_object_clones":39,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":308,"num_read_kb":193213,"num_write":232,"num_write_kb":66787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"325'183","reported_seq":883,"reported_epoch":335,"state":"active+clean","last_fresh":"2023-06-27T12:43:48.554815+0000","last_change":"2023-06-27T12:43:47.529522+0000","last_active":"2023-06-27T12:43:48.554815+0000","last_peered":"2023-06-27T12:43:48.554815+0000","last_clean":"2023-06-27T12:43:48.554815+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:48.554815+0000","last_undegraded":"2023-06-27T12:43:48.554815+0000","last_fullsized":"2023-06-27T12:43:48.554815+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521347,"num_objects":22,"num_object_clones":19,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":127,"num_read_kb":85547,"num_write":112,"num_write_kb":32865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"320'69","reported_seq":835,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.576464+0000","last_change":"2023-06-27T12:43:47.479931+0000","last_active":"2023-06-27T12:43:48.576464+0000","last_peered":"2023-06-27T12:43:48.576464+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:48.576464+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":64,"num_read_kb":47127,"num_write":41,"num_write_kb":12699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":562,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-06-27T12:43:48.559130+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:48.559130+0000","last_peered":"2023-06-27T12:43:48.559130+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:48.559130+0000","last_undegraded":"2023-06-27T12:43:48.559130+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":748,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-06-27T12:43:48.580693+0000","last_change":"2023-06-27T12:43:47.500608+0000","last_active":"2023-06-27T12:43:48.580693+0000","last_peered":"2023-06-27T12:43:48.580693+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:48.580693+0000","last_undegraded":"2023-06-27T12:43:48.580693+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":545,"reported_epoch":335,"state":"active+clean","last_fresh":"2023-06-27T12:43:48.549621+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:48.549621+0000","last_peered":"2023-06-27T12:43:48.549621+0000","last_clean":"2023-06-27T12:43:48.549621+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:48.549621+0000","last_undegraded":"2023-06-27T12:43:48.549621+0000","last_fullsized":"2023-06-27T12:43:48.549621+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"334'188","reported_seq":967,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.550479+0000","last_change":"2023-06-27T12:43:47.618258+0000","last_active":"2023-06-27T12:43:48.550479+0000","last_peered":"2023-06-27T12:43:48.550479+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:48.550479+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5697804,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":77,"num_read_kb":70437,"num_write":119,"num_write_kb":33035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":528,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-06-27T12:43:48.582027+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:48.582027+0000","last_peered":"2023-06-27T12:43:48.582027+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:48.582027+0000","last_undegraded":"2023-06-27T12:43:48.582027+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"329'417","reported_seq":629,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.553604+0000","last_change":"2023-06-27T12:43:47.627889+0000","last_active":"2023-06-27T12:43:48.553604+0000","last_peered":"2023-06-27T12:43:48.553604+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:48.553604+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4370051,"num_objects":22,"num_object_clones":20,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":59,"num_read_kb":33574,"num_write":112,"num_write_kb":34668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":525,"reported_epoch":335,"state":"active+clean","last_fresh":"2023-06-27T12:43:48.573373+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:48.573373+0000","last_peered":"2023-06-27T12:43:48.573373+0000","last_clean":"2023-06-27T12:43:48.573373+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:48.573373+0000","last_undegraded":"2023-06-27T12:43:48.573373+0000","last_fullsized":"2023-06-27T12:43:48.573373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":297,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-06-27T12:43:48.556948+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:48.556948+0000","last_peered":"2023-06-27T12:43:48.556948+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:48.556948+0000","last_undegraded":"2023-06-27T12:43:48.556948+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"320'370","reported_seq":1171,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.576228+0000","last_change":"2023-06-27T12:43:47.473509+0000","last_active":"2023-06-27T12:43:48.576228+0000","last_peered":"2023-06-27T12:43:48.576228+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:48.576228+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521347,"num_objects":49,"num_object_clones":43,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":300,"num_read_kb":178400,"num_write":269,"num_write_kb":78571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":49}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":545,"reported_epoch":335,"state":"active+clean","last_fresh":"2023-06-27T12:43:48.553999+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:48.553999+0000","last_peered":"2023-06-27T12:43:48.553999+0000","last_clean":"2023-06-27T12:43:48.553999+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:48.553999+0000","last_undegraded":"2023-06-27T12:43:48.553999+0000","last_fullsized":"2023-06-27T12:43:48.553999+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"334'237","reported_seq":565,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.575999+0000","last_change":"2023-06-27T12:43:47.507253+0000","last_active":"2023-06-27T12:43:48.575999+0000","last_peered":"2023-06-27T12:43:48.575999+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:48.575999+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8000561,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":53236,"num_write":91,"num_write_kb":25478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":305,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-06-27T12:43:48.551437+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:48.551437+0000","last_peered":"2023-06-27T12:43:48.551437+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:48.551437+0000","last_undegraded":"2023-06-27T12:43:48.551437+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"325'197","reported_seq":1001,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.550774+0000","last_change":"2023-06-27T12:43:47.618904+0000","last_active":"2023-06-27T12:43:48.550774+0000","last_peered":"2023-06-27T12:43:48.550774+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:48.550774+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7361862,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":105,"num_read_kb":112952,"num_write":131,"num_write_kb":37411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"331'122","reported_seq":528,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.578443+0000","last_change":"2023-06-27T12:43:47.477493+0000","last_active":"2023-06-27T12:43:48.578443+0000","last_peered":"2023-06-27T12:43:48.578443+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:48.578443+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5160013,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":56965,"num_write":48,"num_write_kb":13429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":562,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-06-27T12:43:48.556566+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:48.556566+0000","last_peered":"2023-06-27T12:43:48.556566+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:48.556566+0000","last_undegraded":"2023-06-27T12:43:48.556566+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":525,"reported_epoch":335,"state":"active+clean","last_fresh":"2023-06-27T12:43:48.573570+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:48.573570+0000","last_peered":"2023-06-27T12:43:48.573570+0000","last_clean":"2023-06-27T12:43:48.573570+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:48.573570+0000","last_undegraded":"2023-06-27T12:43:48.573570+0000","last_fullsized":"2023-06-27T12:43:48.573570+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"305'220","reported_seq":1006,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:48.554849+0000","last_change":"2023-06-27T12:43:47.611289+0000","last_active":"2023-06-27T12:43:48.554849+0000","last_peered":"2023-06-27T12:43:48.554849+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:48.554849+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832260,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":116,"num_read_kb":120851,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":88929500,"num_objects":336,"num_object_clones":289,"num_object_copies":1008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":1982,"num_read_kb":1442293,"num_write":1773,"num_write_kb":514618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3155,"ondisk_log_size":3155,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673315,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":250648,"kb_used_data":215632,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94121192,"statfs":{"total":96636764160,"available":96380100608,"internally_reserved":0,"allocated":220807168,"data_stored":249908817,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":262116,"kb_used_data":226348,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94109724,"statfs":{"total":96636764160,"available":96368357376,"internally_reserved":0,"allocated":231780352,"data_stored":260342767,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673315,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":284492,"kb_used_data":248724,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94087348,"statfs":{"total":96636764160,"available":96345444352,"internally_reserved":0,"allocated":254693376,"data_stored":288597910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:50.693 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.d 3.b 3.8 3.3 3.0 3.2 3.4 3.7 3.6 2023-06-27T12:43:50.833 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:50.814+0000 7fb2752b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:50.835 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:50.832+0000 7fb2752b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:51.028 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.f] on osd.0 to force-recovery; instructing pg(s) [3.2,3.7,3.8,3.d] on osd.1 to force-recovery; instructing pg(s) [3.0,3.3,3.4,3.6,3.e] on osd.2 to force-recovery; 2023-06-27T12:43:51.028 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:51.564 INFO:tasks.rados.rados.0.smithi138.stdout:1369: done (0 left) 2023-06-27T12:43:51.565 INFO:tasks.rados.rados.0.smithi138.stdout:1370: read oid 40 snap -1 2023-06-27T12:43:51.565 INFO:tasks.rados.rados.0.smithi138.stdout:1370: expect deleted 2023-06-27T12:43:51.565 INFO:tasks.rados.rados.0.smithi138.stdout:1371: snap_create 2023-06-27T12:43:52.568 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:52.576 INFO:tasks.rados.rados.0.smithi138.stdout:1370: done (1 left) 2023-06-27T12:43:52.576 INFO:tasks.rados.rados.0.smithi138.stdout:1371: done (0 left) 2023-06-27T12:43:52.576 INFO:tasks.rados.rados.0.smithi138.stdout:1372: snap_remove snap 156 2023-06-27T12:43:52.707 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:52.702+0000 7f6df11e4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:52.709 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:52.705+0000 7f6df11e4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:52.755 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:52.758 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:52.758 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:53.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~198s 2023-06-27T12:43:53.572 INFO:tasks.rados.rados.0.smithi138.stdout:1372: done (0 left) 2023-06-27T12:43:53.572 INFO:tasks.rados.rados.0.smithi138.stdout:1373: snap_create 2023-06-27T12:43:54.577 INFO:tasks.rados.rados.0.smithi138.stdout:1373: done (0 left) 2023-06-27T12:43:54.577 INFO:tasks.rados.rados.0.smithi138.stdout:1374: write (excl) oid 17 current snap is 161 2023-06-27T12:43:54.577 INFO:tasks.rados.rados.0.smithi138.stdout:1374: seq_num 656 ranges {625392=569702,1637717=745548,2848542=176850} 2023-06-27T12:43:54.583 INFO:tasks.rados.rados.0.smithi138.stdout:1374: writing smithi138138488-17 from 625392 to 1195094 tid 1 2023-06-27T12:43:54.591 INFO:tasks.rados.rados.0.smithi138.stdout:1374: writing smithi138138488-17 from 1637717 to 2383265 tid 2 2023-06-27T12:43:54.593 INFO:tasks.rados.rados.0.smithi138.stdout:1374: writing smithi138138488-17 from 2848542 to 3025392 tid 3 2023-06-27T12:43:54.594 INFO:tasks.rados.rados.0.smithi138.stdout:1375: snap_create 2023-06-27T12:43:54.666 INFO:tasks.rados.rados.0.smithi138.stdout:1374: finishing write tid 1 to smithi138138488-17 2023-06-27T12:43:54.684 INFO:tasks.rados.rados.0.smithi138.stdout:1374: finishing write tid 2 to smithi138138488-17 2023-06-27T12:43:54.718 INFO:tasks.rados.rados.0.smithi138.stdout:1374: finishing write tid 3 to smithi138138488-17 2023-06-27T12:43:54.719 INFO:tasks.rados.rados.0.smithi138.stdout:1374: finishing write tid 4 to smithi138138488-17 2023-06-27T12:43:54.720 INFO:tasks.rados.rados.0.smithi138.stdout:1374: finishing write tid 5 to smithi138138488-17 2023-06-27T12:43:54.720 INFO:tasks.rados.rados.0.smithi138.stdout:1374: oid 17 updating version 0 to 195 2023-06-27T12:43:54.720 INFO:tasks.rados.rados.0.smithi138.stdout:1374: oid 17 version 195 is already newer than 195 2023-06-27T12:43:54.720 INFO:tasks.rados.rados.0.smithi138.stdout:1374: oid 17 version 195 is already newer than 191 2023-06-27T12:43:54.721 INFO:tasks.rados.rados.0.smithi138.stdout:1374: oid 17 version 195 is already newer than 195 2023-06-27T12:43:54.721 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 195 (ObjNum 656 snap 161 seq_num 656) dirty exists 2023-06-27T12:43:54.721 INFO:tasks.rados.rados.0.smithi138.stdout:1374: left oid 17 (ObjNum 656 snap 161 seq_num 656) 2023-06-27T12:43:54.760 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:54.898 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:54.893+0000 7f647749f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:54.901 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:54.897+0000 7f647749f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:54.946 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:54.949 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:54.950 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:55.453 INFO:tasks.rados.rados.0.smithi138.stdout:1374: done (1 left) 2023-06-27T12:43:55.453 INFO:tasks.rados.rados.0.smithi138.stdout:1375: done (0 left) 2023-06-27T12:43:55.453 INFO:tasks.rados.rados.0.smithi138.stdout:1376: delete oid 13 current snap is 162 2023-06-27T12:43:55.482 INFO:tasks.rados.rados.0.smithi138.stdout:1376: done (0 left) 2023-06-27T12:43:55.483 INFO:tasks.rados.rados.0.smithi138.stdout:1377: snap_create 2023-06-27T12:43:56.040 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:43:56.041 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:43:56.041 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:43:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:56.175+0000 7fe2f695f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:56.182+0000 7fe2f695f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:56.378 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:43:56.380 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:43:56.392 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2023-06-27T12:43:55.987504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":91938085,"num_objects":336,"num_object_clones":289,"num_object_copies":1008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":2007,"num_read_kb":1469436,"num_write":1778,"num_write_kb":515873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3262,"ondisk_log_size":3262,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":979672,"kb_used_data":843704,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376507688,"statfs":{"total":386547056640,"available":385543872512,"internally_reserved":0,"allocated":863952896,"data_stored":978513025,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3008585,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":25,"num_read_kb":27143,"num_write":5,"num_write_kb":1255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001228"},"pg_stats":[{"pgid":"3.f","version":"276'37","reported_seq":810,"reported_epoch":339,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:52.842223+0000","last_change":"2023-06-27T12:43:51.582068+0000","last_active":"2023-06-27T12:43:52.842223+0000","last_peered":"2023-06-27T12:43:52.842223+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:52.842223+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3159815,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"316'188","reported_seq":1052,"reported_epoch":339,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:52.826729+0000","last_change":"2023-06-27T12:43:51.592406+0000","last_active":"2023-06-27T12:43:52.826729+0000","last_peered":"2023-06-27T12:43:52.826729+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:52.826729+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":182317,"num_write":137,"num_write_kb":40636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":24}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"325'268","reported_seq":1050,"reported_epoch":340,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:53.583661+0000","last_change":"2023-06-27T12:43:53.589412+0000","last_active":"2023-06-27T12:43:53.583661+0000","last_peered":"2023-06-27T12:43:53.583661+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:53.583661+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8202146,"num_objects":30,"num_object_clones":25,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":229,"num_read_kb":151869,"num_write":169,"num_write_kb":48913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"321'151","reported_seq":991,"reported_epoch":339,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:52.842087+0000","last_change":"2023-06-27T12:43:51.583440+0000","last_active":"2023-06-27T12:43:52.842087+0000","last_peered":"2023-06-27T12:43:52.842087+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:52.842087+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512927,"num_objects":25,"num_object_clones":21,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":152,"num_read_kb":102691,"num_write":110,"num_write_kb":31279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":25}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"331'100","reported_seq":548,"reported_epoch":339,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:52.839253+0000","last_change":"2023-06-27T12:43:51.582632+0000","last_active":"2023-06-27T12:43:52.839253+0000","last_peered":"2023-06-27T12:43:52.839253+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:52.839253+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6773452,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":65312,"num_write":54,"num_write_kb":16369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"334'408","reported_seq":1082,"reported_epoch":339,"state":"active+clean","last_fresh":"2023-06-27T12:43:52.841987+0000","last_change":"2023-06-27T12:43:51.582256+0000","last_active":"2023-06-27T12:43:52.841987+0000","last_peered":"2023-06-27T12:43:52.841987+0000","last_clean":"2023-06-27T12:43:52.841987+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:52.841987+0000","last_undegraded":"2023-06-27T12:43:52.841987+0000","last_fullsized":"2023-06-27T12:43:52.841987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2437147,"num_objects":45,"num_object_clones":39,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":308,"num_read_kb":193213,"num_write":232,"num_write_kb":66787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"325'183","reported_seq":892,"reported_epoch":339,"state":"active+clean","last_fresh":"2023-06-27T12:43:52.836345+0000","last_change":"2023-06-27T12:43:51.584102+0000","last_active":"2023-06-27T12:43:52.836345+0000","last_peered":"2023-06-27T12:43:52.836345+0000","last_clean":"2023-06-27T12:43:52.836345+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:52.836345+0000","last_undegraded":"2023-06-27T12:43:52.836345+0000","last_fullsized":"2023-06-27T12:43:52.836345+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521347,"num_objects":22,"num_object_clones":19,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":127,"num_read_kb":85547,"num_write":112,"num_write_kb":32865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":850,"reported_epoch":340,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:53.578502+0000","last_change":"2023-06-27T12:43:53.583836+0000","last_active":"2023-06-27T12:43:53.578502+0000","last_peered":"2023-06-27T12:43:53.578502+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:53.578502+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":568,"reported_epoch":339,"state":"active+undersized","last_fresh":"2023-06-27T12:43:52.831645+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:52.831645+0000","last_peered":"2023-06-27T12:43:52.831645+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:52.831645+0000","last_undegraded":"2023-06-27T12:43:52.831645+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":758,"reported_epoch":340,"state":"active+undersized","last_fresh":"2023-06-27T12:43:53.576906+0000","last_change":"2023-06-27T12:43:53.588988+0000","last_active":"2023-06-27T12:43:53.576906+0000","last_peered":"2023-06-27T12:43:53.576906+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:53.576906+0000","last_undegraded":"2023-06-27T12:43:53.576906+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":553,"reported_epoch":339,"state":"active+clean","last_fresh":"2023-06-27T12:43:52.836660+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:52.836660+0000","last_peered":"2023-06-27T12:43:52.836660+0000","last_clean":"2023-06-27T12:43:52.836660+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:52.836660+0000","last_undegraded":"2023-06-27T12:43:52.836660+0000","last_fullsized":"2023-06-27T12:43:52.836660+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"334'188","reported_seq":974,"reported_epoch":339,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:52.830261+0000","last_change":"2023-06-27T12:43:51.591604+0000","last_active":"2023-06-27T12:43:52.830261+0000","last_peered":"2023-06-27T12:43:52.830261+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:52.830261+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5697804,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":77,"num_read_kb":70437,"num_write":119,"num_write_kb":33035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":536,"reported_epoch":340,"state":"active+undersized","last_fresh":"2023-06-27T12:43:53.580501+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:53.580501+0000","last_peered":"2023-06-27T12:43:53.580501+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:53.580501+0000","last_undegraded":"2023-06-27T12:43:53.580501+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"329'417","reported_seq":638,"reported_epoch":339,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:52.835282+0000","last_change":"2023-06-27T12:43:51.590749+0000","last_active":"2023-06-27T12:43:52.835282+0000","last_peered":"2023-06-27T12:43:52.835282+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:52.835282+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4370051,"num_objects":22,"num_object_clones":20,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":61,"num_read_kb":37706,"num_write":112,"num_write_kb":34668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":533,"reported_epoch":340,"state":"active+clean","last_fresh":"2023-06-27T12:43:53.579623+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:53.579623+0000","last_peered":"2023-06-27T12:43:53.579623+0000","last_clean":"2023-06-27T12:43:53.579623+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:53.579623+0000","last_undegraded":"2023-06-27T12:43:53.579623+0000","last_fullsized":"2023-06-27T12:43:53.579623+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":305,"reported_epoch":339,"state":"active+undersized","last_fresh":"2023-06-27T12:43:52.843196+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:52.843196+0000","last_peered":"2023-06-27T12:43:52.843196+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:52.843196+0000","last_undegraded":"2023-06-27T12:43:52.843196+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"320'370","reported_seq":1184,"reported_epoch":340,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:53.581611+0000","last_change":"2023-06-27T12:43:53.584397+0000","last_active":"2023-06-27T12:43:53.581611+0000","last_peered":"2023-06-27T12:43:53.581611+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:53.581611+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521347,"num_objects":49,"num_object_clones":43,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":307,"num_read_kb":189206,"num_write":269,"num_write_kb":78571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":49}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":553,"reported_epoch":339,"state":"active+clean","last_fresh":"2023-06-27T12:43:52.842578+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:52.842578+0000","last_peered":"2023-06-27T12:43:52.842578+0000","last_clean":"2023-06-27T12:43:52.842578+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:52.842578+0000","last_undegraded":"2023-06-27T12:43:52.842578+0000","last_fullsized":"2023-06-27T12:43:52.842578+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"334'237","reported_seq":575,"reported_epoch":340,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:53.578969+0000","last_change":"2023-06-27T12:43:53.588321+0000","last_active":"2023-06-27T12:43:53.578969+0000","last_peered":"2023-06-27T12:43:53.578969+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:53.578969+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8000561,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":53236,"num_write":91,"num_write_kb":25478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":311,"reported_epoch":339,"state":"active+undersized","last_fresh":"2023-06-27T12:43:52.835086+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:52.835086+0000","last_peered":"2023-06-27T12:43:52.835086+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:52.835086+0000","last_undegraded":"2023-06-27T12:43:52.835086+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"325'197","reported_seq":1008,"reported_epoch":339,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:52.834104+0000","last_change":"2023-06-27T12:43:51.592617+0000","last_active":"2023-06-27T12:43:52.834104+0000","last_peered":"2023-06-27T12:43:52.834104+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:52.834104+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7361862,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":105,"num_read_kb":112952,"num_write":131,"num_write_kb":37411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"331'122","reported_seq":538,"reported_epoch":340,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:53.575977+0000","last_change":"2023-06-27T12:43:53.583841+0000","last_active":"2023-06-27T12:43:53.575977+0000","last_peered":"2023-06-27T12:43:53.575977+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:53.575977+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5160013,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":56965,"num_write":48,"num_write_kb":13429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":568,"reported_epoch":339,"state":"active+undersized","last_fresh":"2023-06-27T12:43:52.829953+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:52.829953+0000","last_peered":"2023-06-27T12:43:52.829953+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:52.829953+0000","last_undegraded":"2023-06-27T12:43:52.829953+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":533,"reported_epoch":340,"state":"active+clean","last_fresh":"2023-06-27T12:43:53.582009+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:53.582009+0000","last_peered":"2023-06-27T12:43:53.582009+0000","last_clean":"2023-06-27T12:43:53.582009+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:53.582009+0000","last_undegraded":"2023-06-27T12:43:53.582009+0000","last_fullsized":"2023-06-27T12:43:53.582009+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"305'220","reported_seq":1013,"reported_epoch":339,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:52.832131+0000","last_change":"2023-06-27T12:43:51.594143+0000","last_active":"2023-06-27T12:43:52.832131+0000","last_peered":"2023-06-27T12:43:52.832131+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:52.832131+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832260,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":116,"num_read_kb":120851,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":91938085,"num_objects":336,"num_object_clones":289,"num_object_copies":1008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":2007,"num_read_kb":1469436,"num_write":1778,"num_write_kb":515873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3165,"ondisk_log_size":3165,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673320,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":251948,"kb_used_data":216932,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94119892,"statfs":{"total":96636764160,"available":96378769408,"internally_reserved":0,"allocated":222138368,"data_stored":251209954,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706025,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":261396,"kb_used_data":226380,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94110444,"statfs":{"total":96636764160,"available":96369094656,"internally_reserved":0,"allocated":231813120,"data_stored":260360095,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673320,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":285048,"kb_used_data":250032,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94086792,"statfs":{"total":96636764160,"available":96344875008,"internally_reserved":0,"allocated":256032768,"data_stored":289903424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:43:56.395 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.d 3.b 3.2 3.7 3.6 2023-06-27T12:43:56.457 INFO:tasks.rados.rados.0.smithi138.stdout:1377: done (0 left) 2023-06-27T12:43:56.457 INFO:tasks.rados.rados.0.smithi138.stdout:1378: snap_remove snap 153 2023-06-27T12:43:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:56.516+0000 7f78fa2be700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:56.534 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:56.530+0000 7f78fa2be700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:56.802 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.f] on osd.0 to force-backfill; instructing pg(s) [3.2,3.7,3.d] on osd.1 to force-backfill; instructing pg(s) [3.6] on osd.2 to force-backfill; 2023-06-27T12:43:56.802 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:43:56.951 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:43:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:57.073+0000 7f86a90c9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:57.077+0000 7f86a90c9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:57.127 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:57.129 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:57.130 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:57.458 INFO:tasks.rados.rados.0.smithi138.stdout:1378: done (0 left) 2023-06-27T12:43:57.458 INFO:tasks.rados.rados.0.smithi138.stdout:1379: delete oid 1 current snap is 163 2023-06-27T12:43:57.521 INFO:tasks.rados.rados.0.smithi138.stdout:1379: done (0 left) 2023-06-27T12:43:57.521 INFO:tasks.rados.rados.0.smithi138.stdout:1380: delete oid 38 current snap is 163 2023-06-27T12:43:57.571 INFO:tasks.rados.rados.0.smithi138.stdout:1380: done (0 left) 2023-06-27T12:43:57.571 INFO:tasks.rados.rados.0.smithi138.stdout:1381: write (excl) oid 33 current snap is 163 2023-06-27T12:43:57.571 INFO:tasks.rados.rados.0.smithi138.stdout:1381: seq_num 660 ranges {692620=470698,1755292=752886} 2023-06-27T12:43:57.577 INFO:tasks.rados.rados.0.smithi138.stdout:1381: writing smithi138138488-33 from 692620 to 1163318 tid 1 2023-06-27T12:43:57.585 INFO:tasks.rados.rados.0.smithi138.stdout:1381: writing smithi138138488-33 from 1755292 to 2508178 tid 2 2023-06-27T12:43:57.585 INFO:tasks.rados.rados.0.smithi138.stdout:1382: snap_create 2023-06-27T12:43:57.626 INFO:tasks.rados.rados.0.smithi138.stdout:1381: finishing write tid 1 to smithi138138488-33 2023-06-27T12:43:57.673 INFO:tasks.rados.rados.0.smithi138.stdout:1381: finishing write tid 2 to smithi138138488-33 2023-06-27T12:43:57.673 INFO:tasks.rados.rados.0.smithi138.stdout:1381: finishing write tid 3 to smithi138138488-33 2023-06-27T12:43:57.674 INFO:tasks.rados.rados.0.smithi138.stdout:1381: finishing write tid 4 to smithi138138488-33 2023-06-27T12:43:57.675 INFO:tasks.rados.rados.0.smithi138.stdout:1381: oid 33 updating version 0 to 161 2023-06-27T12:43:57.675 INFO:tasks.rados.rados.0.smithi138.stdout:1381: oid 33 updating version 161 to 164 2023-06-27T12:43:57.675 INFO:tasks.rados.rados.0.smithi138.stdout:1381: oid 33 version 164 is already newer than 164 2023-06-27T12:43:57.675 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 164 (ObjNum 660 snap 163 seq_num 660) dirty exists 2023-06-27T12:43:57.675 INFO:tasks.rados.rados.0.smithi138.stdout:1381: left oid 33 (ObjNum 660 snap 163 seq_num 660) 2023-06-27T12:43:58.464 INFO:tasks.rados.rados.0.smithi138.stdout:1381: done (1 left) 2023-06-27T12:43:58.464 INFO:tasks.rados.rados.0.smithi138.stdout:1382: done (0 left) 2023-06-27T12:43:58.465 INFO:tasks.rados.rados.0.smithi138.stdout:1383: read oid 17 snap -1 2023-06-27T12:43:58.465 INFO:tasks.rados.rados.0.smithi138.stdout:1383: expect (ObjNum 656 snap 161 seq_num 656) 2023-06-27T12:43:58.465 INFO:tasks.rados.rados.0.smithi138.stdout:1384: delete oid 23 current snap is 164 2023-06-27T12:43:58.492 INFO:tasks.rados.rados.0.smithi138.stdout:1384: done (1 left) 2023-06-27T12:43:58.493 INFO:tasks.rados.rados.0.smithi138.stdout:1385: write oid 4 current snap is 164 2023-06-27T12:43:58.493 INFO:tasks.rados.rados.0.smithi138.stdout:1385: seq_num 662 ranges {726234=737549,1988609=756555} 2023-06-27T12:43:58.501 INFO:tasks.rados.rados.0.smithi138.stdout:1385: writing smithi138138488-4 from 726234 to 1463783 tid 1 2023-06-27T12:43:58.509 INFO:tasks.rados.rados.0.smithi138.stdout:1385: writing smithi138138488-4 from 1988609 to 2745164 tid 2 2023-06-27T12:43:58.509 INFO:tasks.rados.rados.0.smithi138.stdout:1386: delete oid 22 current snap is 164 2023-06-27T12:43:58.509 INFO:tasks.rados.rados.0.smithi138.stdout:1383: expect (ObjNum 656 snap 161 seq_num 656) 2023-06-27T12:43:58.667 INFO:tasks.rados.rados.0.smithi138.stdout:1385: finishing write tid 1 to smithi138138488-4 2023-06-27T12:43:58.667 INFO:tasks.rados.rados.0.smithi138.stdout:1385: finishing write tid 2 to smithi138138488-4 2023-06-27T12:43:58.667 INFO:tasks.rados.rados.0.smithi138.stdout:1385: finishing write tid 3 to smithi138138488-4 2023-06-27T12:43:58.667 INFO:tasks.rados.rados.0.smithi138.stdout:1385: finishing write tid 4 to smithi138138488-4 2023-06-27T12:43:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1385: oid 4 updating version 0 to 243 2023-06-27T12:43:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1385: oid 4 version 243 is already newer than 242 2023-06-27T12:43:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1385: oid 4 version 243 is already newer than 243 2023-06-27T12:43:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 243 (ObjNum 662 snap 164 seq_num 662) dirty exists 2023-06-27T12:43:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1385: left oid 4 (ObjNum 662 snap 164 seq_num 662) 2023-06-27T12:43:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1383: done (2 left) 2023-06-27T12:43:58.668 INFO:tasks.rados.rados.0.smithi138.stdout:1385: done (1 left) 2023-06-27T12:43:58.669 INFO:tasks.rados.rados.0.smithi138.stdout:1386: done (0 left) 2023-06-27T12:43:58.669 INFO:tasks.rados.rados.0.smithi138.stdout:1387: write oid 8 current snap is 164 2023-06-27T12:43:58.669 INFO:tasks.rados.rados.0.smithi138.stdout:1387: seq_num 664 ranges {759848=488047,1989220=760224} 2023-06-27T12:43:58.673 INFO:tasks.rados.rados.0.smithi138.stdout:1387: writing smithi138138488-8 from 759848 to 1247895 tid 1 2023-06-27T12:43:58.682 INFO:tasks.rados.rados.0.smithi138.stdout:1387: writing smithi138138488-8 from 1989220 to 2749444 tid 2 2023-06-27T12:43:58.682 INFO:tasks.rados.rados.0.smithi138.stdout:1388: snap_remove snap 159 2023-06-27T12:43:58.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~204s 2023-06-27T12:43:59.131 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:43:59.256 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:59.251+0000 7f71930e7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:59.259 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:43:59.255+0000 7f71930e7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:43:59.305 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:43:59.308 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:43:59.308 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:43:59.468 INFO:tasks.rados.rados.0.smithi138.stdout:1388: done (1 left) 2023-06-27T12:43:59.469 INFO:tasks.rados.rados.0.smithi138.stdout:1389: read oid 15 snap -1 2023-06-27T12:43:59.469 INFO:tasks.rados.rados.0.smithi138.stdout:1389: expect (ObjNum 650 snap 156 seq_num 650) 2023-06-27T12:43:59.470 INFO:tasks.rados.rados.0.smithi138.stdout:1390: read oid 3 snap -1 2023-06-27T12:43:59.470 INFO:tasks.rados.rados.0.smithi138.stdout:1390: expect (ObjNum 638 snap 153 seq_num 638) 2023-06-27T12:43:59.470 INFO:tasks.rados.rados.0.smithi138.stdout:1391: snap_remove snap 157 2023-06-27T12:44:00.473 INFO:tasks.rados.rados.0.smithi138.stdout:1391: done (3 left) 2023-06-27T12:44:00.473 INFO:tasks.rados.rados.0.smithi138.stdout:1392: snap_create 2023-06-27T12:44:00.473 INFO:tasks.rados.rados.0.smithi138.stdout:1387: finishing write tid 1 to smithi138138488-8 2023-06-27T12:44:00.473 INFO:tasks.rados.rados.0.smithi138.stdout:1387: finishing write tid 2 to smithi138138488-8 2023-06-27T12:44:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1387: finishing write tid 3 to smithi138138488-8 2023-06-27T12:44:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1387: finishing write tid 4 to smithi138138488-8 2023-06-27T12:44:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1387: oid 8 updating version 0 to 104 2023-06-27T12:44:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1387: oid 8 version 104 is already newer than 104 2023-06-27T12:44:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1387: oid 8 version 104 is already newer than 104 2023-06-27T12:44:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 8 v 104 (ObjNum 664 snap 164 seq_num 664) dirty exists 2023-06-27T12:44:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1387: left oid 8 (ObjNum 664 snap 164 seq_num 664) 2023-06-27T12:44:00.475 INFO:tasks.rados.rados.0.smithi138.stdout:1389: expect (ObjNum 650 snap 156 seq_num 650) 2023-06-27T12:44:00.586 INFO:tasks.rados.rados.0.smithi138.stdout:1390: expect (ObjNum 638 snap 153 seq_num 638) 2023-06-27T12:44:01.309 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:01.437 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:01.432+0000 7fa0e75dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:01.440 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:01.436+0000 7fa0e75dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:01.474 INFO:tasks.rados.rados.0.smithi138.stdout:1387: done (3 left) 2023-06-27T12:44:01.475 INFO:tasks.rados.rados.0.smithi138.stdout:1389: done (2 left) 2023-06-27T12:44:01.475 INFO:tasks.rados.rados.0.smithi138.stdout:1390: done (1 left) 2023-06-27T12:44:01.475 INFO:tasks.rados.rados.0.smithi138.stdout:1392: done (0 left) 2023-06-27T12:44:01.476 INFO:tasks.rados.rados.0.smithi138.stdout:1393: read oid 43 snap -1 2023-06-27T12:44:01.476 INFO:tasks.rados.rados.0.smithi138.stdout:1393: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:44:01.476 INFO:tasks.rados.rados.0.smithi138.stdout:1394: read oid 5 snap -1 2023-06-27T12:44:01.476 INFO:tasks.rados.rados.0.smithi138.stdout:1394: expect (ObjNum 549 snap 130 seq_num 549) 2023-06-27T12:44:01.476 INFO:tasks.rados.rados.0.smithi138.stdout:1395: read oid 48 snap -1 2023-06-27T12:44:01.477 INFO:tasks.rados.rados.0.smithi138.stdout:1395: expect (ObjNum 602 snap 144 seq_num 602) 2023-06-27T12:44:01.477 INFO:tasks.rados.rados.0.smithi138.stdout:1396: delete oid 16 current snap is 165 2023-06-27T12:44:01.488 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:01.492 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:01.492 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:01.511 INFO:tasks.rados.rados.0.smithi138.stdout:1396: done (3 left) 2023-06-27T12:44:01.512 INFO:tasks.rados.rados.0.smithi138.stdout:1397: snap_remove snap 163 2023-06-27T12:44:01.815 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:01.815 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:01.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:01.943 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:01.936+0000 7ff239c22700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:01.946 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:01.942+0000 7ff239c22700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:02.137 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:02.139 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:02.150 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2023-06-27T12:44:01.989297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82576322,"num_objects":343,"num_object_clones":296,"num_object_copies":1029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":276,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":27,"num_read":2018,"num_read_kb":1474125,"num_write":1795,"num_write_kb":521208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3305,"ondisk_log_size":3305,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":988096,"kb_used_data":852128,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376499264,"statfs":{"total":386547056640,"available":385535246336,"internally_reserved":0,"allocated":872579072,"data_stored":989572392,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9361763,"num_objects":7,"num_object_clones":7,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":11,"num_read_kb":4689,"num_write":17,"num_write_kb":5335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001289"},"pg_stats":[{"pgid":"3.f","version":"344'39","reported_seq":819,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.684030+0000","last_change":"2023-06-27T12:43:57.481459+0000","last_active":"2023-06-27T12:43:58.684030+0000","last_peered":"2023-06-27T12:43:58.684030+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:43:58.684030+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"341'195","reported_seq":1071,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.491232+0000","last_change":"2023-06-27T12:43:57.836156+0000","last_active":"2023-06-27T12:43:58.491232+0000","last_peered":"2023-06-27T12:43:58.491232+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:43:58.491232+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12656100,"num_objects":25,"num_object_clones":21,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":148,"num_read_kb":187003,"num_write":142,"num_write_kb":42095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"325'268","reported_seq":1057,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.474357+0000","last_change":"2023-06-27T12:43:57.494561+0000","last_active":"2023-06-27T12:43:58.474357+0000","last_peered":"2023-06-27T12:43:58.474357+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:43:58.474357+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8202146,"num_objects":30,"num_object_clones":25,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":229,"num_read_kb":151869,"num_write":169,"num_write_kb":48913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"344'164","reported_seq":1003,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.686545+0000","last_change":"2023-06-27T12:43:57.491917+0000","last_active":"2023-06-27T12:43:58.686545+0000","last_peered":"2023-06-27T12:43:58.686545+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:43:58.686545+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050751,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":154,"num_read_kb":102692,"num_write":114,"num_write_kb":32475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":560,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.698258+0000","last_change":"2023-06-27T12:43:57.485511+0000","last_active":"2023-06-27T12:43:58.698258+0000","last_peered":"2023-06-27T12:43:58.698258+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:43:58.698258+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":65313,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"334'408","reported_seq":1091,"reported_epoch":345,"state":"active+clean","last_fresh":"2023-06-27T12:43:58.682300+0000","last_change":"2023-06-27T12:43:57.486383+0000","last_active":"2023-06-27T12:43:58.682300+0000","last_peered":"2023-06-27T12:43:58.682300+0000","last_clean":"2023-06-27T12:43:58.682300+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:43:58.682300+0000","last_undegraded":"2023-06-27T12:43:58.682300+0000","last_fullsized":"2023-06-27T12:43:58.682300+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2437147,"num_objects":45,"num_object_clones":39,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":308,"num_read_kb":193213,"num_write":232,"num_write_kb":66787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"325'183","reported_seq":900,"reported_epoch":345,"state":"active+clean","last_fresh":"2023-06-27T12:43:58.685148+0000","last_change":"2023-06-27T12:43:57.491778+0000","last_active":"2023-06-27T12:43:58.685148+0000","last_peered":"2023-06-27T12:43:58.685148+0000","last_clean":"2023-06-27T12:43:58.685148+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:43:58.685148+0000","last_undegraded":"2023-06-27T12:43:58.685148+0000","last_fullsized":"2023-06-27T12:43:58.685148+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521347,"num_objects":22,"num_object_clones":19,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":127,"num_read_kb":85547,"num_write":112,"num_write_kb":32865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":857,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.474081+0000","last_change":"2023-06-27T12:43:57.484942+0000","last_active":"2023-06-27T12:43:58.474081+0000","last_peered":"2023-06-27T12:43:58.474081+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:43:58.474081+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":578,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-06-27T12:43:58.474301+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:43:58.474301+0000","last_peered":"2023-06-27T12:43:58.474301+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:43:58.474301+0000","last_undegraded":"2023-06-27T12:43:58.474301+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":765,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-06-27T12:43:58.475847+0000","last_change":"2023-06-27T12:43:57.493538+0000","last_active":"2023-06-27T12:43:58.475847+0000","last_peered":"2023-06-27T12:43:58.475847+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:43:58.475847+0000","last_undegraded":"2023-06-27T12:43:58.475847+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":559,"reported_epoch":345,"state":"active+clean","last_fresh":"2023-06-27T12:43:58.681604+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:43:58.681604+0000","last_peered":"2023-06-27T12:43:58.681604+0000","last_clean":"2023-06-27T12:43:58.681604+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:43:58.681604+0000","last_undegraded":"2023-06-27T12:43:58.681604+0000","last_fullsized":"2023-06-27T12:43:58.681604+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"334'188","reported_seq":988,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.475940+0000","last_change":"2023-06-27T12:43:57.829014+0000","last_active":"2023-06-27T12:43:58.475940+0000","last_peered":"2023-06-27T12:43:58.475940+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:43:58.475940+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5697804,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":78,"num_read_kb":70437,"num_write":119,"num_write_kb":33035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":21}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":542,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-06-27T12:43:58.473548+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:43:58.473548+0000","last_peered":"2023-06-27T12:43:58.473548+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:43:58.473548+0000","last_undegraded":"2023-06-27T12:43:58.473548+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"329'417","reported_seq":652,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.473618+0000","last_change":"2023-06-27T12:43:57.827895+0000","last_active":"2023-06-27T12:43:58.473618+0000","last_peered":"2023-06-27T12:43:58.473618+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:43:58.473618+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4370051,"num_objects":22,"num_object_clones":20,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":61,"num_read_kb":37706,"num_write":112,"num_write_kb":34668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":539,"reported_epoch":345,"state":"active+clean","last_fresh":"2023-06-27T12:43:58.476556+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:43:58.476556+0000","last_peered":"2023-06-27T12:43:58.476556+0000","last_clean":"2023-06-27T12:43:58.476556+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:43:58.476556+0000","last_undegraded":"2023-06-27T12:43:58.476556+0000","last_fullsized":"2023-06-27T12:43:58.476556+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":311,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-06-27T12:43:58.682564+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:43:58.682564+0000","last_peered":"2023-06-27T12:43:58.682564+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:43:58.682564+0000","last_undegraded":"2023-06-27T12:43:58.682564+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"344'381","reported_seq":1192,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.472912+0000","last_change":"2023-06-27T12:43:57.488739+0000","last_active":"2023-06-27T12:43:58.472912+0000","last_peered":"2023-06-27T12:43:58.472912+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:43:58.472912+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3983358,"num_objects":50,"num_object_clones":44,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":308,"num_read_kb":189206,"num_write":269,"num_write_kb":78571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":50}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":559,"reported_epoch":345,"state":"active+clean","last_fresh":"2023-06-27T12:43:58.682917+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:43:58.682917+0000","last_peered":"2023-06-27T12:43:58.682917+0000","last_clean":"2023-06-27T12:43:58.682917+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:43:58.682917+0000","last_undegraded":"2023-06-27T12:43:58.682917+0000","last_fullsized":"2023-06-27T12:43:58.682917+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"345'243","reported_seq":586,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.549680+0000","last_change":"2023-06-27T12:43:57.521729+0000","last_active":"2023-06-27T12:43:58.549680+0000","last_peered":"2023-06-27T12:43:58.549680+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:43:58.549680+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8924946,"num_objects":17,"num_object_clones":17,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":53237,"num_write":95,"num_write_kb":26938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":17}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":321,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-06-27T12:43:58.472065+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:43:58.472065+0000","last_peered":"2023-06-27T12:43:58.472065+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:43:58.472065+0000","last_undegraded":"2023-06-27T12:43:58.472065+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"325'197","reported_seq":1020,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.477012+0000","last_change":"2023-06-27T12:43:57.822436+0000","last_active":"2023-06-27T12:43:58.477012+0000","last_peered":"2023-06-27T12:43:58.477012+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:43:58.477012+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7361862,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":105,"num_read_kb":112952,"num_write":131,"num_write_kb":37411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"331'122","reported_seq":546,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.473800+0000","last_change":"2023-06-27T12:43:57.490259+0000","last_active":"2023-06-27T12:43:58.473800+0000","last_peered":"2023-06-27T12:43:58.473800+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:43:58.473800+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5160013,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":56965,"num_write":48,"num_write_kb":13429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":578,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-06-27T12:43:58.477407+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:43:58.477407+0000","last_peered":"2023-06-27T12:43:58.477407+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:43:58.477407+0000","last_undegraded":"2023-06-27T12:43:58.477407+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":539,"reported_epoch":345,"state":"active+clean","last_fresh":"2023-06-27T12:43:58.472660+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:43:58.472660+0000","last_peered":"2023-06-27T12:43:58.472660+0000","last_clean":"2023-06-27T12:43:58.472660+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:43:58.472660+0000","last_undegraded":"2023-06-27T12:43:58.472660+0000","last_fullsized":"2023-06-27T12:43:58.472660+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"305'220","reported_seq":1026,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:43:58.476425+0000","last_change":"2023-06-27T12:43:57.820574+0000","last_active":"2023-06-27T12:43:58.476425+0000","last_peered":"2023-06-27T12:43:58.476425+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:43:58.476425+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117814,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":117,"num_read_kb":120851,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":82576322,"num_objects":343,"num_object_clones":296,"num_object_copies":1029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":276,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":27,"num_read":2018,"num_read_kb":1474125,"num_write":1795,"num_write_kb":521208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3208,"ondisk_log_size":3208,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673325,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":255896,"kb_used_data":220880,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94115944,"statfs":{"total":96636764160,"available":96374726656,"internally_reserved":0,"allocated":226181120,"data_stored":256553624,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":264120,"kb_used_data":229104,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94107720,"statfs":{"total":96636764160,"available":96366305280,"internally_reserved":0,"allocated":234602496,"data_stored":264107724,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673325,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":286800,"kb_used_data":251784,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94085040,"statfs":{"total":96636764160,"available":96343080960,"internally_reserved":0,"allocated":257826816,"data_stored":291871492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:02.481 INFO:tasks.rados.rados.0.smithi138.stdout:1397: done (3 left) 2023-06-27T12:44:02.481 INFO:tasks.rados.rados.0.smithi138.stdout:1398: read oid 8 snap -1 2023-06-27T12:44:02.482 INFO:tasks.rados.rados.0.smithi138.stdout:1398: expect (ObjNum 664 snap 164 seq_num 664) 2023-06-27T12:44:02.482 INFO:tasks.rados.rados.0.smithi138.stdout:1399: read oid 23 snap -1 2023-06-27T12:44:02.482 INFO:tasks.rados.rados.0.smithi138.stdout:1399: expect deleted 2023-06-27T12:44:02.482 INFO:tasks.rados.rados.0.smithi138.stdout:1400: write (excl) oid 9 current snap is 165 2023-06-27T12:44:02.482 INFO:tasks.rados.rados.0.smithi138.stdout:1400: seq_num 666 ranges {793462=638545,2106184=763893} 2023-06-27T12:44:02.491 INFO:tasks.rados.rados.0.smithi138.stdout:1400: writing smithi138138488-9 from 793462 to 1432007 tid 1 2023-06-27T12:44:02.502 INFO:tasks.rados.rados.0.smithi138.stdout:1400: writing smithi138138488-9 from 2106184 to 2870077 tid 2 2023-06-27T12:44:02.503 INFO:tasks.rados.rados.0.smithi138.stdout:1401: write (excl) oid 6 current snap is 165 2023-06-27T12:44:02.503 INFO:tasks.rados.rados.0.smithi138.stdout:1393: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:44:02.679 INFO:tasks.rados.rados.0.smithi138.stdout:1395: expect (ObjNum 602 snap 144 seq_num 602) 2023-06-27T12:44:02.810 INFO:tasks.rados.rados.0.smithi138.stdout:1394: expect (ObjNum 549 snap 130 seq_num 549) 2023-06-27T12:44:02.929 INFO:tasks.rados.rados.0.smithi138.stdout:1398: expect (ObjNum 664 snap 164 seq_num 664) 2023-06-27T12:44:03.111 INFO:tasks.rados.rados.0.smithi138.stdout:1400: finishing write tid 1 to smithi138138488-9 2023-06-27T12:44:03.111 INFO:tasks.rados.rados.0.smithi138.stdout:1400: finishing write tid 2 to smithi138138488-9 2023-06-27T12:44:03.111 INFO:tasks.rados.rados.0.smithi138.stdout:1400: finishing write tid 3 to smithi138138488-9 2023-06-27T12:44:03.112 INFO:tasks.rados.rados.0.smithi138.stdout:1400: finishing write tid 4 to smithi138138488-9 2023-06-27T12:44:03.112 INFO:tasks.rados.rados.0.smithi138.stdout:1400: oid 9 updating version 0 to 387 2023-06-27T12:44:03.112 INFO:tasks.rados.rados.0.smithi138.stdout:1400: oid 9 version 387 is already newer than 384 2023-06-27T12:44:03.112 INFO:tasks.rados.rados.0.smithi138.stdout:1400: oid 9 version 387 is already newer than 387 2023-06-27T12:44:03.112 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 387 (ObjNum 666 snap 165 seq_num 666) dirty exists 2023-06-27T12:44:03.112 INFO:tasks.rados.rados.0.smithi138.stdout:1400: left oid 9 (ObjNum 666 snap 165 seq_num 666) 2023-06-27T12:44:03.112 INFO:tasks.rados.rados.0.smithi138.stdout:1401: seq_num 667 ranges {410269=713794,1648313=507551,2917489=292780} 2023-06-27T12:44:03.117 INFO:tasks.rados.rados.0.smithi138.stdout:1401: writing smithi138138488-6 from 410269 to 1124063 tid 1 2023-06-27T12:44:03.122 INFO:tasks.rados.rados.0.smithi138.stdout:1401: writing smithi138138488-6 from 1648313 to 2155864 tid 2 2023-06-27T12:44:03.129 INFO:tasks.rados.rados.0.smithi138.stdout:1401: writing smithi138138488-6 from 2917489 to 3210269 tid 3 2023-06-27T12:44:03.129 INFO:tasks.rados.rados.0.smithi138.stdout:1393: done (6 left) 2023-06-27T12:44:03.129 INFO:tasks.rados.rados.0.smithi138.stdout:1394: done (5 left) 2023-06-27T12:44:03.129 INFO:tasks.rados.rados.0.smithi138.stdout:1395: done (4 left) 2023-06-27T12:44:03.129 INFO:tasks.rados.rados.0.smithi138.stdout:1398: done (3 left) 2023-06-27T12:44:03.129 INFO:tasks.rados.rados.0.smithi138.stdout:1399: done (2 left) 2023-06-27T12:44:03.130 INFO:tasks.rados.rados.0.smithi138.stdout:1400: done (1 left) 2023-06-27T12:44:03.130 INFO:tasks.rados.rados.0.smithi138.stdout:1402: read oid 22 snap 160 2023-06-27T12:44:03.130 INFO:tasks.rados.rados.0.smithi138.stdout:1402: expect (ObjNum 578 snap 138 seq_num 578) 2023-06-27T12:44:03.130 INFO:tasks.rados.rados.0.smithi138.stdout:1403: read oid 33 snap 164 2023-06-27T12:44:03.130 INFO:tasks.rados.rados.0.smithi138.stdout:1403: expect (ObjNum 660 snap 163 seq_num 660) 2023-06-27T12:44:03.130 INFO:tasks.rados.rados.0.smithi138.stdout:1404: write oid 36 current snap is 165 2023-06-27T12:44:03.130 INFO:tasks.rados.rados.0.smithi138.stdout:1404: seq_num 668 ranges {427076=789043,1706795=767562,2944912=282164} 2023-06-27T12:44:03.136 INFO:tasks.rados.rados.0.smithi138.stdout:1404: writing smithi138138488-36 from 427076 to 1216119 tid 1 2023-06-27T12:44:03.143 INFO:tasks.rados.rados.0.smithi138.stdout:1404: writing smithi138138488-36 from 1706795 to 2474357 tid 2 2023-06-27T12:44:03.148 INFO:tasks.rados.rados.0.smithi138.stdout:1404: writing smithi138138488-36 from 2944912 to 3227076 tid 3 2023-06-27T12:44:03.148 INFO:tasks.rados.rados.0.smithi138.stdout:1405: read oid 44 snap -1 2023-06-27T12:44:03.148 INFO:tasks.rados.rados.0.smithi138.stdout:1405: expect deleted 2023-06-27T12:44:03.148 INFO:tasks.rados.rados.0.smithi138.stdout:1406: write (excl) oid 2 current snap is 165 2023-06-27T12:44:03.148 INFO:tasks.rados.rados.0.smithi138.stdout:1406: seq_num 669 ranges {443883=464292,1365277=511220,2572335=671548} 2023-06-27T12:44:03.154 INFO:tasks.rados.rados.0.smithi138.stdout:1406: writing smithi138138488-2 from 443883 to 908175 tid 1 2023-06-27T12:44:03.160 INFO:tasks.rados.rados.0.smithi138.stdout:1406: writing smithi138138488-2 from 1365277 to 1876497 tid 2 2023-06-27T12:44:03.168 INFO:tasks.rados.rados.0.smithi138.stdout:1406: writing smithi138138488-2 from 2572335 to 3243883 tid 3 2023-06-27T12:44:03.169 INFO:tasks.rados.rados.0.smithi138.stdout:1407: write oid 9 current snap is 165 2023-06-27T12:44:03.169 INFO:tasks.rados.rados.0.smithi138.stdout:1407: seq_num 670 ranges {460690=655894,1540112=771231,2716111=539778} 2023-06-27T12:44:03.176 INFO:tasks.rados.rados.0.smithi138.stdout:1407: writing smithi138138488-9 from 460690 to 1116584 tid 1 2023-06-27T12:44:03.184 INFO:tasks.rados.rados.0.smithi138.stdout:1407: writing smithi138138488-9 from 1540112 to 2311343 tid 2 2023-06-27T12:44:03.191 INFO:tasks.rados.rados.0.smithi138.stdout:1407: writing smithi138138488-9 from 2716111 to 3255889 tid 3 2023-06-27T12:44:03.191 INFO:tasks.rados.rados.0.smithi138.stdout:1408: write (excl) oid 10 current snap is 165 2023-06-27T12:44:03.191 INFO:tasks.rados.rados.0.smithi138.stdout:1408: seq_num 671 ranges {477497=731143,1998594=514889,3143534=133963} 2023-06-27T12:44:03.199 INFO:tasks.rados.rados.0.smithi138.stdout:1408: writing smithi138138488-10 from 477497 to 1208640 tid 1 2023-06-27T12:44:03.206 INFO:tasks.rados.rados.0.smithi138.stdout:1408: writing smithi138138488-10 from 1998594 to 2513483 tid 2 2023-06-27T12:44:03.211 INFO:tasks.rados.rados.0.smithi138.stdout:1408: writing smithi138138488-10 from 3143534 to 3277497 tid 3 2023-06-27T12:44:03.211 INFO:tasks.rados.rados.0.smithi138.stdout:1409: write oid 41 current snap is 165 2023-06-27T12:44:03.212 INFO:tasks.rados.rados.0.smithi138.stdout:1402: expect (ObjNum 578 snap 138 seq_num 578) 2023-06-27T12:44:03.451 INFO:tasks.rados.rados.0.smithi138.stdout:1401: finishing write tid 1 to smithi138138488-6 2023-06-27T12:44:03.451 INFO:tasks.rados.rados.0.smithi138.stdout:1403: expect (ObjNum 660 snap 163 seq_num 660) 2023-06-27T12:44:03.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:03.620 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:03.615+0000 7fc871f2c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:03.623 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:03.619+0000 7fc871f2c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:03.663 INFO:tasks.rados.rados.0.smithi138.stdout:1404: finishing write tid 1 to smithi138138488-36 2023-06-27T12:44:03.663 INFO:tasks.rados.rados.0.smithi138.stdout:1406: finishing write tid 1 to smithi138138488-2 2023-06-27T12:44:03.664 INFO:tasks.rados.rados.0.smithi138.stdout:1404: finishing write tid 2 to smithi138138488-36 2023-06-27T12:44:03.664 INFO:tasks.rados.rados.0.smithi138.stdout:1401: finishing write tid 2 to smithi138138488-6 2023-06-27T12:44:03.664 INFO:tasks.rados.rados.0.smithi138.stdout:1401: finishing write tid 3 to smithi138138488-6 2023-06-27T12:44:03.664 INFO:tasks.rados.rados.0.smithi138.stdout:1401: finishing write tid 4 to smithi138138488-6 2023-06-27T12:44:03.664 INFO:tasks.rados.rados.0.smithi138.stdout:1401: finishing write tid 5 to smithi138138488-6 2023-06-27T12:44:03.664 INFO:tasks.rados.rados.0.smithi138.stdout:1401: oid 6 updating version 0 to 194 2023-06-27T12:44:03.664 INFO:tasks.rados.rados.0.smithi138.stdout:1401: oid 6 version 194 is already newer than 194 2023-06-27T12:44:03.665 INFO:tasks.rados.rados.0.smithi138.stdout:1401: oid 6 version 194 is already newer than 190 2023-06-27T12:44:03.665 INFO:tasks.rados.rados.0.smithi138.stdout:1401: oid 6 version 194 is already newer than 194 2023-06-27T12:44:03.665 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 194 (ObjNum 667 snap 165 seq_num 667) dirty exists 2023-06-27T12:44:03.665 INFO:tasks.rados.rados.0.smithi138.stdout:1401: left oid 6 (ObjNum 667 snap 165 seq_num 667) 2023-06-27T12:44:03.665 INFO:tasks.rados.rados.0.smithi138.stdout:1404: finishing write tid 3 to smithi138138488-36 2023-06-27T12:44:03.665 INFO:tasks.rados.rados.0.smithi138.stdout:1406: finishing write tid 2 to smithi138138488-2 2023-06-27T12:44:03.665 INFO:tasks.rados.rados.0.smithi138.stdout:1406: finishing write tid 3 to smithi138138488-2 2023-06-27T12:44:03.666 INFO:tasks.rados.rados.0.smithi138.stdout:1406: finishing write tid 4 to smithi138138488-2 2023-06-27T12:44:03.666 INFO:tasks.rados.rados.0.smithi138.stdout:1404: finishing write tid 4 to smithi138138488-36 2023-06-27T12:44:03.666 INFO:tasks.rados.rados.0.smithi138.stdout:1406: finishing write tid 5 to smithi138138488-2 2023-06-27T12:44:03.666 INFO:tasks.rados.rados.0.smithi138.stdout:1406: oid 2 updating version 0 to 425 2023-06-27T12:44:03.666 INFO:tasks.rados.rados.0.smithi138.stdout:1406: oid 2 version 425 is already newer than 425 2023-06-27T12:44:03.666 INFO:tasks.rados.rados.0.smithi138.stdout:1406: oid 2 version 425 is already newer than 425 2023-06-27T12:44:03.666 INFO:tasks.rados.rados.0.smithi138.stdout:1406: oid 2 version 425 is already newer than 421 2023-06-27T12:44:03.667 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 425 (ObjNum 669 snap 165 seq_num 669) dirty exists 2023-06-27T12:44:03.667 INFO:tasks.rados.rados.0.smithi138.stdout:1406: left oid 2 (ObjNum 669 snap 165 seq_num 669) 2023-06-27T12:44:03.667 INFO:tasks.rados.rados.0.smithi138.stdout:1404: finishing write tid 5 to smithi138138488-36 2023-06-27T12:44:03.667 INFO:tasks.rados.rados.0.smithi138.stdout:1404: oid 36 updating version 0 to 273 2023-06-27T12:44:03.667 INFO:tasks.rados.rados.0.smithi138.stdout:1404: oid 36 version 273 is already newer than 273 2023-06-27T12:44:03.667 INFO:tasks.rados.rados.0.smithi138.stdout:1404: oid 36 version 273 is already newer than 273 2023-06-27T12:44:03.667 INFO:tasks.rados.rados.0.smithi138.stdout:1404: oid 36 version 273 is already newer than 273 2023-06-27T12:44:03.668 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 273 (ObjNum 668 snap 165 seq_num 668) dirty exists 2023-06-27T12:44:03.668 INFO:tasks.rados.rados.0.smithi138.stdout:1404: left oid 36 (ObjNum 668 snap 165 seq_num 668) 2023-06-27T12:44:03.668 INFO:tasks.rados.rados.0.smithi138.stdout:1407: finishing write tid 1 to smithi138138488-9 2023-06-27T12:44:03.668 INFO:tasks.rados.rados.0.smithi138.stdout:1407: finishing write tid 2 to smithi138138488-9 2023-06-27T12:44:03.668 INFO:tasks.rados.rados.0.smithi138.stdout:1408: finishing write tid 1 to smithi138138488-10 2023-06-27T12:44:03.668 INFO:tasks.rados.rados.0.smithi138.stdout:1407: finishing write tid 3 to smithi138138488-9 2023-06-27T12:44:03.668 INFO:tasks.rados.rados.0.smithi138.stdout:1407: finishing write tid 4 to smithi138138488-9 2023-06-27T12:44:03.669 INFO:tasks.rados.rados.0.smithi138.stdout:1407: finishing write tid 5 to smithi138138488-9 2023-06-27T12:44:03.669 INFO:tasks.rados.rados.0.smithi138.stdout:1407: oid 9 updating version 0 to 391 2023-06-27T12:44:03.669 INFO:tasks.rados.rados.0.smithi138.stdout:1407: oid 9 version 391 is already newer than 391 2023-06-27T12:44:03.669 INFO:tasks.rados.rados.0.smithi138.stdout:1407: oid 9 version 391 is already newer than 391 2023-06-27T12:44:03.669 INFO:tasks.rados.rados.0.smithi138.stdout:1407: oid 9 version 391 is already newer than 391 2023-06-27T12:44:03.669 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 391 (ObjNum 670 snap 165 seq_num 670) dirty exists 2023-06-27T12:44:03.669 INFO:tasks.rados.rados.0.smithi138.stdout:1407: left oid 9 (ObjNum 670 snap 165 seq_num 670) 2023-06-27T12:44:03.670 INFO:tasks.rados.rados.0.smithi138.stdout:1408: finishing write tid 2 to smithi138138488-10 2023-06-27T12:44:03.670 INFO:tasks.rados.rados.0.smithi138.stdout:1408: finishing write tid 3 to smithi138138488-10 2023-06-27T12:44:03.670 INFO:tasks.rados.rados.0.smithi138.stdout:1408: finishing write tid 4 to smithi138138488-10 2023-06-27T12:44:03.670 INFO:tasks.rados.rados.0.smithi138.stdout:1408: finishing write tid 5 to smithi138138488-10 2023-06-27T12:44:03.670 INFO:tasks.rados.rados.0.smithi138.stdout:1408: oid 10 updating version 0 to 187 2023-06-27T12:44:03.670 INFO:tasks.rados.rados.0.smithi138.stdout:1408: oid 10 updating version 187 to 191 2023-06-27T12:44:03.670 INFO:tasks.rados.rados.0.smithi138.stdout:1408: oid 10 version 191 is already newer than 191 2023-06-27T12:44:03.670 INFO:tasks.rados.rados.0.smithi138.stdout:1408: oid 10 version 191 is already newer than 191 2023-06-27T12:44:03.671 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 191 (ObjNum 671 snap 165 seq_num 671) dirty exists 2023-06-27T12:44:03.671 INFO:tasks.rados.rados.0.smithi138.stdout:1408: left oid 10 (ObjNum 671 snap 165 seq_num 671) 2023-06-27T12:44:03.671 INFO:tasks.rados.rados.0.smithi138.stdout:1409: seq_num 672 ranges {494304=406392,1540723=774900,2770957=523347} 2023-06-27T12:44:03.671 INFO:tasks.rados.rados.0.smithi138.stdout:1409: writing smithi138138488-41 from 494304 to 900696 tid 1 2023-06-27T12:44:03.672 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:03.674 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:03.674 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:03.675 INFO:tasks.rados.rados.0.smithi138.stdout:1409: writing smithi138138488-41 from 1540723 to 2315623 tid 2 2023-06-27T12:44:03.681 INFO:tasks.rados.rados.0.smithi138.stdout:1409: writing smithi138138488-41 from 2770957 to 3294304 tid 3 2023-06-27T12:44:03.681 INFO:tasks.rados.rados.0.smithi138.stdout:1401: done (8 left) 2023-06-27T12:44:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:1402: done (7 left) 2023-06-27T12:44:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:1403: done (6 left) 2023-06-27T12:44:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:1404: done (5 left) 2023-06-27T12:44:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:1405: done (4 left) 2023-06-27T12:44:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:1406: done (3 left) 2023-06-27T12:44:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:1407: done (2 left) 2023-06-27T12:44:03.682 INFO:tasks.rados.rados.0.smithi138.stdout:1408: done (1 left) 2023-06-27T12:44:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:1410: read oid 47 snap -1 2023-06-27T12:44:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:1410: expect (ObjNum 463 snap 108 seq_num 463) 2023-06-27T12:44:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:1411: write (excl) oid 40 current snap is 165 2023-06-27T12:44:03.683 INFO:tasks.rados.rados.0.smithi138.stdout:1411: seq_num 673 ranges {511111=481641,1599205=518558,2682027=489947} 2023-06-27T12:44:03.687 INFO:tasks.rados.rados.0.smithi138.stdout:1411: writing smithi138138488-40 from 511111 to 992752 tid 1 2023-06-27T12:44:03.693 INFO:tasks.rados.rados.0.smithi138.stdout:1411: writing smithi138138488-40 from 1599205 to 2117763 tid 2 2023-06-27T12:44:03.699 INFO:tasks.rados.rados.0.smithi138.stdout:1411: writing smithi138138488-40 from 2682027 to 3171974 tid 3 2023-06-27T12:44:03.700 INFO:tasks.rados.rados.0.smithi138.stdout:1412: write oid 25 current snap is 165 2023-06-27T12:44:03.700 INFO:tasks.rados.rados.0.smithi138.stdout:1409: finishing write tid 1 to smithi138138488-41 2023-06-27T12:44:03.700 INFO:tasks.rados.rados.0.smithi138.stdout:1412: seq_num 674 ranges {527918=556890,1657687=778569,3225803=102115} 2023-06-27T12:44:03.706 INFO:tasks.rados.rados.0.smithi138.stdout:1412: writing smithi138138488-25 from 527918 to 1084808 tid 1 2023-06-27T12:44:03.715 INFO:tasks.rados.rados.0.smithi138.stdout:1412: writing smithi138138488-25 from 1657687 to 2436256 tid 2 2023-06-27T12:44:03.721 INFO:tasks.rados.rados.0.smithi138.stdout:1412: writing smithi138138488-25 from 3225803 to 3327918 tid 3 2023-06-27T12:44:03.721 INFO:tasks.rados.rados.0.smithi138.stdout:1413: snap_remove snap 161 2023-06-27T12:44:04.471 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~209s 2023-06-27T12:44:04.485 INFO:tasks.rados.rados.0.smithi138.stdout:1413: done (4 left) 2023-06-27T12:44:04.486 INFO:tasks.rados.rados.0.smithi138.stdout:1414: snap_create 2023-06-27T12:44:04.486 INFO:tasks.rados.rados.0.smithi138.stdout:1409: finishing write tid 2 to smithi138138488-41 2023-06-27T12:44:04.486 INFO:tasks.rados.rados.0.smithi138.stdout:1410: expect (ObjNum 463 snap 108 seq_num 463) 2023-06-27T12:44:04.681 INFO:tasks.rados.rados.0.smithi138.stdout:1409: finishing write tid 3 to smithi138138488-41 2023-06-27T12:44:04.682 INFO:tasks.rados.rados.0.smithi138.stdout:1409: finishing write tid 4 to smithi138138488-41 2023-06-27T12:44:04.682 INFO:tasks.rados.rados.0.smithi138.stdout:1409: finishing write tid 5 to smithi138138488-41 2023-06-27T12:44:04.682 INFO:tasks.rados.rados.0.smithi138.stdout:1409: oid 41 updating version 0 to 205 2023-06-27T12:44:04.682 INFO:tasks.rados.rados.0.smithi138.stdout:1409: oid 41 version 205 is already newer than 203 2023-06-27T12:44:04.682 INFO:tasks.rados.rados.0.smithi138.stdout:1409: oid 41 version 205 is already newer than 205 2023-06-27T12:44:04.683 INFO:tasks.rados.rados.0.smithi138.stdout:1409: oid 41 version 205 is already newer than 205 2023-06-27T12:44:04.683 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 41 v 205 (ObjNum 672 snap 165 seq_num 672) dirty exists 2023-06-27T12:44:04.683 INFO:tasks.rados.rados.0.smithi138.stdout:1409: left oid 41 (ObjNum 672 snap 165 seq_num 672) 2023-06-27T12:44:04.683 INFO:tasks.rados.rados.0.smithi138.stdout:1411: finishing write tid 1 to smithi138138488-40 2023-06-27T12:44:04.683 INFO:tasks.rados.rados.0.smithi138.stdout:1412: finishing write tid 1 to smithi138138488-25 2023-06-27T12:44:04.683 INFO:tasks.rados.rados.0.smithi138.stdout:1411: finishing write tid 2 to smithi138138488-40 2023-06-27T12:44:04.683 INFO:tasks.rados.rados.0.smithi138.stdout:1411: finishing write tid 3 to smithi138138488-40 2023-06-27T12:44:04.683 INFO:tasks.rados.rados.0.smithi138.stdout:1411: finishing write tid 4 to smithi138138488-40 2023-06-27T12:44:04.684 INFO:tasks.rados.rados.0.smithi138.stdout:1411: finishing write tid 5 to smithi138138488-40 2023-06-27T12:44:04.684 INFO:tasks.rados.rados.0.smithi138.stdout:1411: oid 40 updating version 0 to 172 2023-06-27T12:44:04.684 INFO:tasks.rados.rados.0.smithi138.stdout:1411: oid 40 version 172 is already newer than 172 2023-06-27T12:44:04.684 INFO:tasks.rados.rados.0.smithi138.stdout:1411: oid 40 version 172 is already newer than 169 2023-06-27T12:44:04.684 INFO:tasks.rados.rados.0.smithi138.stdout:1411: oid 40 version 172 is already newer than 172 2023-06-27T12:44:04.684 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 172 (ObjNum 673 snap 165 seq_num 673) dirty exists 2023-06-27T12:44:04.684 INFO:tasks.rados.rados.0.smithi138.stdout:1411: left oid 40 (ObjNum 673 snap 165 seq_num 673) 2023-06-27T12:44:04.684 INFO:tasks.rados.rados.0.smithi138.stdout:1412: finishing write tid 2 to smithi138138488-25 2023-06-27T12:44:04.685 INFO:tasks.rados.rados.0.smithi138.stdout:1412: finishing write tid 3 to smithi138138488-25 2023-06-27T12:44:04.685 INFO:tasks.rados.rados.0.smithi138.stdout:1412: finishing write tid 4 to smithi138138488-25 2023-06-27T12:44:04.685 INFO:tasks.rados.rados.0.smithi138.stdout:1412: finishing write tid 5 to smithi138138488-25 2023-06-27T12:44:04.685 INFO:tasks.rados.rados.0.smithi138.stdout:1412: oid 25 updating version 0 to 131 2023-06-27T12:44:04.685 INFO:tasks.rados.rados.0.smithi138.stdout:1412: oid 25 version 131 is already newer than 131 2023-06-27T12:44:04.685 INFO:tasks.rados.rados.0.smithi138.stdout:1412: oid 25 version 131 is already newer than 131 2023-06-27T12:44:04.685 INFO:tasks.rados.rados.0.smithi138.stdout:1412: oid 25 version 131 is already newer than 131 2023-06-27T12:44:04.685 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 131 (ObjNum 674 snap 165 seq_num 674) dirty exists 2023-06-27T12:44:04.686 INFO:tasks.rados.rados.0.smithi138.stdout:1412: left oid 25 (ObjNum 674 snap 165 seq_num 674) 2023-06-27T12:44:05.488 INFO:tasks.rados.rados.0.smithi138.stdout:1409: done (4 left) 2023-06-27T12:44:05.489 INFO:tasks.rados.rados.0.smithi138.stdout:1410: done (3 left) 2023-06-27T12:44:05.489 INFO:tasks.rados.rados.0.smithi138.stdout:1411: done (2 left) 2023-06-27T12:44:05.489 INFO:tasks.rados.rados.0.smithi138.stdout:1412: done (1 left) 2023-06-27T12:44:05.489 INFO:tasks.rados.rados.0.smithi138.stdout:1414: done (0 left) 2023-06-27T12:44:05.489 INFO:tasks.rados.rados.0.smithi138.stdout:1415: snap_create 2023-06-27T12:44:05.676 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:05.807 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:05.802+0000 7f845e1e5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:05.810 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:05.806+0000 7f845e1e5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:05.858 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:05.859 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:06.497 INFO:tasks.rados.rados.0.smithi138.stdout:1415: done (0 left) 2023-06-27T12:44:06.497 INFO:tasks.rados.rados.0.smithi138.stdout:1416: delete oid 25 current snap is 167 2023-06-27T12:44:06.537 INFO:tasks.rados.rados.0.smithi138.stdout:1416: done (0 left) 2023-06-27T12:44:06.538 INFO:tasks.rados.rados.0.smithi138.stdout:1417: write oid 35 current snap is 167 2023-06-27T12:44:06.538 INFO:tasks.rados.rados.0.smithi138.stdout:1417: seq_num 676 ranges {561532=707388,2058298=782238} 2023-06-27T12:44:06.544 INFO:tasks.rados.rados.0.smithi138.stdout:1417: writing smithi138138488-35 from 561532 to 1268920 tid 1 2023-06-27T12:44:06.553 INFO:tasks.rados.rados.0.smithi138.stdout:1417: writing smithi138138488-35 from 2058298 to 2840536 tid 2 2023-06-27T12:44:06.553 INFO:tasks.rados.rados.0.smithi138.stdout:1418: read oid 12 snap -1 2023-06-27T12:44:06.553 INFO:tasks.rados.rados.0.smithi138.stdout:1418: expect (ObjNum 551 snap 132 seq_num 551) 2023-06-27T12:44:06.553 INFO:tasks.rados.rados.0.smithi138.stdout:1419: write (excl) oid 10 current snap is 167 2023-06-27T12:44:06.553 INFO:tasks.rados.rados.0.smithi138.stdout:1419: seq_num 677 ranges {578339=498990,1833133=525896,2791719=586620} 2023-06-27T12:44:06.558 INFO:tasks.rados.rados.0.smithi138.stdout:1419: writing smithi138138488-10 from 578339 to 1077329 tid 1 2023-06-27T12:44:06.565 INFO:tasks.rados.rados.0.smithi138.stdout:1419: writing smithi138138488-10 from 1833133 to 2359029 tid 2 2023-06-27T12:44:06.571 INFO:tasks.rados.rados.0.smithi138.stdout:1419: writing smithi138138488-10 from 2791719 to 3378339 tid 3 2023-06-27T12:44:06.572 INFO:tasks.rados.rados.0.smithi138.stdout:1420: write oid 2 current snap is 167 2023-06-27T12:44:06.572 INFO:tasks.rados.rados.0.smithi138.stdout:1420: seq_num 678 ranges {595146=574239,1891615=669554,3219142=176004} 2023-06-27T12:44:06.578 INFO:tasks.rados.rados.0.smithi138.stdout:1420: writing smithi138138488-2 from 595146 to 1169385 tid 1 2023-06-27T12:44:06.586 INFO:tasks.rados.rados.0.smithi138.stdout:1420: writing smithi138138488-2 from 1891615 to 2561169 tid 2 2023-06-27T12:44:06.591 INFO:tasks.rados.rados.0.smithi138.stdout:1420: writing smithi138138488-2 from 3219142 to 3395146 tid 3 2023-06-27T12:44:06.591 INFO:tasks.rados.rados.0.smithi138.stdout:1421: write oid 40 current snap is 167 2023-06-27T12:44:06.591 INFO:tasks.rados.rados.0.smithi138.stdout:1421: seq_num 679 ranges {611953=649488,1950097=529565,3246565=165388} 2023-06-27T12:44:06.598 INFO:tasks.rados.rados.0.smithi138.stdout:1421: writing smithi138138488-40 from 611953 to 1261441 tid 1 2023-06-27T12:44:06.605 INFO:tasks.rados.rados.0.smithi138.stdout:1421: writing smithi138138488-40 from 1950097 to 2479662 tid 2 2023-06-27T12:44:06.611 INFO:tasks.rados.rados.0.smithi138.stdout:1421: writing smithi138138488-40 from 3246565 to 3411953 tid 3 2023-06-27T12:44:06.611 INFO:tasks.rados.rados.0.smithi138.stdout:1422: read oid 1 snap -1 2023-06-27T12:44:06.611 INFO:tasks.rados.rados.0.smithi138.stdout:1422: expect deleted 2023-06-27T12:44:06.611 INFO:tasks.rados.rados.0.smithi138.stdout:1423: delete oid 26 current snap is 167 2023-06-27T12:44:06.612 INFO:tasks.rados.rados.0.smithi138.stdout:1417: finishing write tid 1 to smithi138138488-35 2023-06-27T12:44:06.639 INFO:tasks.rados.rados.0.smithi138.stdout:1418: expect (ObjNum 551 snap 132 seq_num 551) 2023-06-27T12:44:06.865 INFO:tasks.rados.rados.0.smithi138.stdout:1417: finishing write tid 2 to smithi138138488-35 2023-06-27T12:44:06.866 INFO:tasks.rados.rados.0.smithi138.stdout:1418: done (6 left) 2023-06-27T12:44:06.866 INFO:tasks.rados.rados.0.smithi138.stdout:1423: done (5 left) 2023-06-27T12:44:06.866 INFO:tasks.rados.rados.0.smithi138.stdout:1424: write oid 7 current snap is 167 2023-06-27T12:44:06.867 INFO:tasks.rados.rados.0.smithi138.stdout:1417: finishing write tid 3 to smithi138138488-35 2023-06-27T12:44:06.867 INFO:tasks.rados.rados.0.smithi138.stdout:1417: finishing write tid 4 to smithi138138488-35 2023-06-27T12:44:06.867 INFO:tasks.rados.rados.0.smithi138.stdout:1417: oid 35 updating version 0 to 431 2023-06-27T12:44:06.867 INFO:tasks.rados.rados.0.smithi138.stdout:1417: oid 35 version 431 is already newer than 431 2023-06-27T12:44:06.867 INFO:tasks.rados.rados.0.smithi138.stdout:1417: oid 35 version 431 is already newer than 431 2023-06-27T12:44:06.867 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 431 (ObjNum 676 snap 167 seq_num 676) dirty exists 2023-06-27T12:44:06.867 INFO:tasks.rados.rados.0.smithi138.stdout:1417: left oid 35 (ObjNum 676 snap 167 seq_num 676) 2023-06-27T12:44:06.868 INFO:tasks.rados.rados.0.smithi138.stdout:1419: finishing write tid 1 to smithi138138488-10 2023-06-27T12:44:06.868 INFO:tasks.rados.rados.0.smithi138.stdout:1420: finishing write tid 1 to smithi138138488-2 2023-06-27T12:44:06.868 INFO:tasks.rados.rados.0.smithi138.stdout:1420: finishing write tid 2 to smithi138138488-2 2023-06-27T12:44:06.868 INFO:tasks.rados.rados.0.smithi138.stdout:1420: finishing write tid 3 to smithi138138488-2 2023-06-27T12:44:06.868 INFO:tasks.rados.rados.0.smithi138.stdout:1420: finishing write tid 4 to smithi138138488-2 2023-06-27T12:44:06.868 INFO:tasks.rados.rados.0.smithi138.stdout:1420: finishing write tid 5 to smithi138138488-2 2023-06-27T12:44:06.868 INFO:tasks.rados.rados.0.smithi138.stdout:1420: oid 2 updating version 0 to 436 2023-06-27T12:44:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:1420: oid 2 version 436 is already newer than 436 2023-06-27T12:44:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:1420: oid 2 version 436 is already newer than 436 2023-06-27T12:44:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:1420: oid 2 version 436 is already newer than 436 2023-06-27T12:44:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 436 (ObjNum 678 snap 167 seq_num 678) dirty exists 2023-06-27T12:44:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:1420: left oid 2 (ObjNum 678 snap 167 seq_num 678) 2023-06-27T12:44:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:1421: finishing write tid 1 to smithi138138488-40 2023-06-27T12:44:06.869 INFO:tasks.rados.rados.0.smithi138.stdout:1421: finishing write tid 2 to smithi138138488-40 2023-06-27T12:44:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:1421: finishing write tid 3 to smithi138138488-40 2023-06-27T12:44:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:1419: finishing write tid 2 to smithi138138488-10 2023-06-27T12:44:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:1419: finishing write tid 3 to smithi138138488-10 2023-06-27T12:44:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:1419: finishing write tid 4 to smithi138138488-10 2023-06-27T12:44:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:1421: finishing write tid 4 to smithi138138488-40 2023-06-27T12:44:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:1421: finishing write tid 5 to smithi138138488-40 2023-06-27T12:44:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:1421: oid 40 updating version 0 to 178 2023-06-27T12:44:06.870 INFO:tasks.rados.rados.0.smithi138.stdout:1421: oid 40 updating version 178 to 179 2023-06-27T12:44:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:1421: oid 40 version 179 is already newer than 179 2023-06-27T12:44:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:1421: oid 40 version 179 is already newer than 179 2023-06-27T12:44:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 179 (ObjNum 679 snap 167 seq_num 679) dirty exists 2023-06-27T12:44:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:1421: left oid 40 (ObjNum 679 snap 167 seq_num 679) 2023-06-27T12:44:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:1419: finishing write tid 5 to smithi138138488-10 2023-06-27T12:44:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:1419: oid 10 updating version 0 to 198 2023-06-27T12:44:06.871 INFO:tasks.rados.rados.0.smithi138.stdout:1419: oid 10 version 198 is already newer than 198 2023-06-27T12:44:06.872 INFO:tasks.rados.rados.0.smithi138.stdout:1419: oid 10 version 198 is already newer than 198 2023-06-27T12:44:06.872 INFO:tasks.rados.rados.0.smithi138.stdout:1419: oid 10 version 198 is already newer than 194 2023-06-27T12:44:06.872 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 198 (ObjNum 677 snap 167 seq_num 677) dirty exists 2023-06-27T12:44:06.872 INFO:tasks.rados.rados.0.smithi138.stdout:1419: left oid 10 (ObjNum 677 snap 167 seq_num 677) 2023-06-27T12:44:06.872 INFO:tasks.rados.rados.0.smithi138.stdout:1424: seq_num 681 ranges {645567=799986,1950708=533234,3185058=260509} 2023-06-27T12:44:06.873 INFO:tasks.rados.rados.0.smithi138.stdout:1424: writing smithi138138488-7 from 645567 to 1445553 tid 1 2023-06-27T12:44:06.878 INFO:tasks.rados.rados.0.smithi138.stdout:1424: writing smithi138138488-7 from 1950708 to 2483942 tid 2 2023-06-27T12:44:06.884 INFO:tasks.rados.rados.0.smithi138.stdout:1424: writing smithi138138488-7 from 3185058 to 3445567 tid 3 2023-06-27T12:44:06.884 INFO:tasks.rados.rados.0.smithi138.stdout:1417: done (5 left) 2023-06-27T12:44:06.884 INFO:tasks.rados.rados.0.smithi138.stdout:1419: done (4 left) 2023-06-27T12:44:06.884 INFO:tasks.rados.rados.0.smithi138.stdout:1420: done (3 left) 2023-06-27T12:44:06.884 INFO:tasks.rados.rados.0.smithi138.stdout:1421: done (2 left) 2023-06-27T12:44:06.884 INFO:tasks.rados.rados.0.smithi138.stdout:1422: done (1 left) 2023-06-27T12:44:06.885 INFO:tasks.rados.rados.0.smithi138.stdout:1425: read oid 28 snap -1 2023-06-27T12:44:06.885 INFO:tasks.rados.rados.0.smithi138.stdout:1425: expect deleted 2023-06-27T12:44:06.885 INFO:tasks.rados.rados.0.smithi138.stdout:1426: snap_remove snap 166 2023-06-27T12:44:07.154 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:07.154 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:07.154 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:07.282 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:07.275+0000 7fed42559700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:07.285 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:07.281+0000 7fed42559700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:07.477 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:07.478 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:07.490 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2023-06-27T12:44:06.492201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":90744854,"num_objects":349,"num_object_clones":302,"num_object_copies":1047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":2071,"num_read_kb":1528363,"num_write":1839,"num_write_kb":535265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3373,"ondisk_log_size":3373,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1017156,"kb_used_data":880524,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376470204,"statfs":{"total":386547056640,"available":385505488896,"internally_reserved":0,"allocated":901656576,"data_stored":1025237238,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8168532,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":53,"num_read_kb":54238,"num_write":44,"num_write_kb":14057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.502904"},"pg_stats":[{"pgid":"3.f","version":"344'39","reported_seq":830,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:02.491214+0000","last_change":"2023-06-27T12:44:02.512000+0000","last_active":"2023-06-27T12:44:02.491214+0000","last_peered":"2023-06-27T12:44:02.491214+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:02.491214+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"341'195","reported_seq":1082,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:03.694166+0000","last_change":"2023-06-27T12:44:02.507337+0000","last_active":"2023-06-27T12:44:03.694166+0000","last_peered":"2023-06-27T12:44:03.694166+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:03.694166+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12656100,"num_objects":25,"num_object_clones":21,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":194391,"num_write":142,"num_write_kb":42095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"349'273","reported_seq":1069,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:03.203338+0000","last_change":"2023-06-27T12:44:02.543619+0000","last_active":"2023-06-27T12:44:03.203338+0000","last_peered":"2023-06-27T12:44:03.203338+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:03.203338+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689549,"num_objects":31,"num_object_clones":26,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":230,"num_read_kb":151870,"num_write":174,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":31}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"349'172","reported_seq":1022,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:03.754157+0000","last_change":"2023-06-27T12:44:02.521386+0000","last_active":"2023-06-27T12:44:03.754157+0000","last_peered":"2023-06-27T12:44:03.754157+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:03.754157+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9361862,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":163,"num_read_kb":111756,"num_write":119,"num_write_kb":33932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":577,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:02.539341+0000","last_change":"2023-06-27T12:44:02.564947+0000","last_active":"2023-06-27T12:44:02.539341+0000","last_peered":"2023-06-27T12:44:02.539341+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:02.539341+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"334'408","reported_seq":1103,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-06-27T12:44:02.485997+0000","last_change":"2023-06-27T12:44:02.507707+0000","last_active":"2023-06-27T12:44:02.485997+0000","last_peered":"2023-06-27T12:44:02.485997+0000","last_clean":"2023-06-27T12:44:02.485997+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:02.485997+0000","last_undegraded":"2023-06-27T12:44:02.485997+0000","last_fullsized":"2023-06-27T12:44:02.485997+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2437147,"num_objects":45,"num_object_clones":39,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":309,"num_read_kb":193213,"num_write":232,"num_write_kb":66787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"349'191","reported_seq":916,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-06-27T12:44:03.342931+0000","last_change":"2023-06-27T12:44:02.534874+0000","last_active":"2023-06-27T12:44:03.342931+0000","last_peered":"2023-06-27T12:44:03.342931+0000","last_clean":"2023-06-27T12:44:03.342931+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:03.342931+0000","last_undegraded":"2023-06-27T12:44:03.342931+0000","last_fullsized":"2023-06-27T12:44:03.342931+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7092785,"num_objects":23,"num_object_clones":20,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":129,"num_read_kb":85548,"num_write":117,"num_write_kb":34214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":864,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:02.517282+0000","last_change":"2023-06-27T12:44:02.520716+0000","last_active":"2023-06-27T12:44:02.517282+0000","last_peered":"2023-06-27T12:44:02.517282+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:02.517282+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":584,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-06-27T12:44:02.499402+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:02.499402+0000","last_peered":"2023-06-27T12:44:02.499402+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:02.499402+0000","last_undegraded":"2023-06-27T12:44:02.499402+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":772,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-06-27T12:44:02.534250+0000","last_change":"2023-06-27T12:44:02.544904+0000","last_active":"2023-06-27T12:44:02.534250+0000","last_peered":"2023-06-27T12:44:02.534250+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:02.534250+0000","last_undegraded":"2023-06-27T12:44:02.534250+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":567,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-06-27T12:44:02.486225+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:02.486225+0000","last_peered":"2023-06-27T12:44:02.486225+0000","last_clean":"2023-06-27T12:44:02.486225+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:02.486225+0000","last_undegraded":"2023-06-27T12:44:02.486225+0000","last_fullsized":"2023-06-27T12:44:02.486225+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"349'194","reported_seq":1002,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:03.174208+0000","last_change":"2023-06-27T12:44:02.504905+0000","last_active":"2023-06-27T12:44:03.174208+0000","last_peered":"2023-06-27T12:44:03.174208+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:03.174208+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6101205,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":80,"num_read_kb":70438,"num_write":124,"num_write_kb":34515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":546,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-06-27T12:44:02.526686+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:02.526686+0000","last_peered":"2023-06-27T12:44:02.526686+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:02.526686+0000","last_undegraded":"2023-06-27T12:44:02.526686+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"349'425","reported_seq":666,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:03.227455+0000","last_change":"2023-06-27T12:44:02.506586+0000","last_active":"2023-06-27T12:44:03.227455+0000","last_peered":"2023-06-27T12:44:03.227455+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:03.227455+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4773419,"num_objects":23,"num_object_clones":21,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":63,"num_read_kb":37707,"num_write":117,"num_write_kb":36278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":543,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-06-27T12:44:02.505047+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:02.505047+0000","last_peered":"2023-06-27T12:44:02.505047+0000","last_clean":"2023-06-27T12:44:02.505047+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:02.505047+0000","last_undegraded":"2023-06-27T12:44:02.505047+0000","last_fullsized":"2023-06-27T12:44:02.505047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":319,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-06-27T12:44:02.494432+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:02.494432+0000","last_peered":"2023-06-27T12:44:02.494432+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:02.494432+0000","last_undegraded":"2023-06-27T12:44:02.494432+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"349'391","reported_seq":1208,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:03.334656+0000","last_change":"2023-06-27T12:44:02.514557+0000","last_active":"2023-06-27T12:44:03.334656+0000","last_peered":"2023-06-27T12:44:03.334656+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:03.334656+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260690,"num_objects":51,"num_object_clones":45,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":311,"num_read_kb":189208,"num_write":278,"num_write_kb":81864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":567,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-06-27T12:44:02.500734+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:02.500734+0000","last_peered":"2023-06-27T12:44:02.500734+0000","last_clean":"2023-06-27T12:44:02.500734+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:02.500734+0000","last_undegraded":"2023-06-27T12:44:02.500734+0000","last_fullsized":"2023-06-27T12:44:02.500734+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"345'243","reported_seq":596,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:02.588186+0000","last_change":"2023-06-27T12:44:02.588186+0000","last_active":"2023-06-27T12:44:02.588186+0000","last_peered":"2023-06-27T12:44:02.588186+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:02.588186+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8924946,"num_objects":17,"num_object_clones":17,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":59441,"num_write":95,"num_write_kb":26938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":17}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":327,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-06-27T12:44:02.489706+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:02.489706+0000","last_peered":"2023-06-27T12:44:02.489706+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:02.489706+0000","last_undegraded":"2023-06-27T12:44:02.489706+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"349'205","reported_seq":1038,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:03.686953+0000","last_change":"2023-06-27T12:44:02.502666+0000","last_active":"2023-06-27T12:44:03.686953+0000","last_peered":"2023-06-27T12:44:03.686953+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:03.686953+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10656166,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":121604,"num_write":136,"num_write_kb":39077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"349'131","reported_seq":558,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:03.723787+0000","last_change":"2023-06-27T12:44:02.513736+0000","last_active":"2023-06-27T12:44:03.723787+0000","last_peered":"2023-06-27T12:44:03.723787+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:03.723787+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5580188,"num_objects":9,"num_object_clones":9,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":99,"num_read_kb":56966,"num_write":53,"num_write_kb":14834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":584,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-06-27T12:44:02.497930+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:02.497930+0000","last_peered":"2023-06-27T12:44:02.497930+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:02.497930+0000","last_undegraded":"2023-06-27T12:44:02.497930+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":543,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-06-27T12:44:02.519075+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:02.519075+0000","last_peered":"2023-06-27T12:44:02.519075+0000","last_clean":"2023-06-27T12:44:02.519075+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:02.519075+0000","last_undegraded":"2023-06-27T12:44:02.519075+0000","last_fullsized":"2023-06-27T12:44:02.519075+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"345'226","reported_seq":1039,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:03.140850+0000","last_change":"2023-06-27T12:44:02.588896+0000","last_active":"2023-06-27T12:44:03.140850+0000","last_peered":"2023-06-27T12:44:03.140850+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:03.140850+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117814,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":121,"num_read_kb":132245,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":90744854,"num_objects":349,"num_object_clones":302,"num_object_copies":1047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":25,"num_read":2071,"num_read_kb":1528363,"num_write":1839,"num_write_kb":535265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3276,"ondisk_log_size":3276,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673330,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":263544,"kb_used_data":228528,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94108296,"statfs":{"total":96636764160,"available":96366895104,"internally_reserved":0,"allocated":234012672,"data_stored":266141975,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":276004,"kb_used_data":240988,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94095836,"statfs":{"total":96636764160,"available":96354136064,"internally_reserved":0,"allocated":246771712,"data_stored":278367226,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673330,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":296328,"kb_used_data":260648,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94075512,"statfs":{"total":96636764160,"available":96333324288,"internally_reserved":0,"allocated":266903552,"data_stored":303688485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:07.500 INFO:tasks.rados.rados.0.smithi138.stdout:1426: done (2 left) 2023-06-27T12:44:07.500 INFO:tasks.rados.rados.0.smithi138.stdout:1427: write (excl) oid 4 current snap is 167 2023-06-27T12:44:07.500 INFO:tasks.rados.rados.0.smithi138.stdout:1424: finishing write tid 1 to smithi138138488-7 2023-06-27T12:44:07.500 INFO:tasks.rados.rados.0.smithi138.stdout:1427: seq_num 682 ranges {662374=475235,1609190=676892,2812481=624101} 2023-06-27T12:44:07.506 INFO:tasks.rados.rados.0.smithi138.stdout:1427: writing smithi138138488-4 from 662374 to 1137609 tid 1 2023-06-27T12:44:07.512 INFO:tasks.rados.rados.0.smithi138.stdout:1427: writing smithi138138488-4 from 1609190 to 2286082 tid 2 2023-06-27T12:44:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:1427: writing smithi138138488-4 from 2812481 to 3436582 tid 3 2023-06-27T12:44:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:1428: delete oid 47 current snap is 167 2023-06-27T12:44:07.519 INFO:tasks.rados.rados.0.smithi138.stdout:1424: finishing write tid 2 to smithi138138488-7 2023-06-27T12:44:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:1424: finishing write tid 3 to smithi138138488-7 2023-06-27T12:44:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:1424: finishing write tid 4 to smithi138138488-7 2023-06-27T12:44:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:1424: finishing write tid 5 to smithi138138488-7 2023-06-27T12:44:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:1424: oid 7 updating version 0 to 203 2023-06-27T12:44:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:1424: oid 7 version 203 is already newer than 203 2023-06-27T12:44:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:1424: oid 7 version 203 is already newer than 201 2023-06-27T12:44:07.520 INFO:tasks.rados.rados.0.smithi138.stdout:1424: oid 7 version 203 is already newer than 203 2023-06-27T12:44:07.521 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 203 (ObjNum 681 snap 167 seq_num 681) dirty exists 2023-06-27T12:44:07.521 INFO:tasks.rados.rados.0.smithi138.stdout:1424: left oid 7 (ObjNum 681 snap 167 seq_num 681) 2023-06-27T12:44:07.562 INFO:tasks.rados.rados.0.smithi138.stdout:1427: finishing write tid 1 to smithi138138488-4 2023-06-27T12:44:07.598 INFO:tasks.rados.rados.0.smithi138.stdout:1424: done (3 left) 2023-06-27T12:44:07.599 INFO:tasks.rados.rados.0.smithi138.stdout:1425: done (2 left) 2023-06-27T12:44:07.599 INFO:tasks.rados.rados.0.smithi138.stdout:1428: done (1 left) 2023-06-27T12:44:07.599 INFO:tasks.rados.rados.0.smithi138.stdout:1429: delete oid 37 current snap is 167 2023-06-27T12:44:07.601 INFO:tasks.rados.rados.0.smithi138.stdout:1427: finishing write tid 2 to smithi138138488-4 2023-06-27T12:44:07.642 INFO:tasks.rados.rados.0.smithi138.stdout:1427: finishing write tid 3 to smithi138138488-4 2023-06-27T12:44:07.642 INFO:tasks.rados.rados.0.smithi138.stdout:1429: done (1 left) 2023-06-27T12:44:07.643 INFO:tasks.rados.rados.0.smithi138.stdout:1430: write (excl) oid 41 current snap is 167 2023-06-27T12:44:07.643 INFO:tasks.rados.rados.0.smithi138.stdout:1427: finishing write tid 4 to smithi138138488-4 2023-06-27T12:44:07.643 INFO:tasks.rados.rados.0.smithi138.stdout:1430: seq_num 685 ranges {712795=417335,1784636=540572,2894750=457917} 2023-06-27T12:44:07.647 INFO:tasks.rados.rados.0.smithi138.stdout:1430: writing smithi138138488-41 from 712795 to 1130130 tid 1 2023-06-27T12:44:07.654 INFO:tasks.rados.rados.0.smithi138.stdout:1430: writing smithi138138488-41 from 1784636 to 2325208 tid 2 2023-06-27T12:44:07.660 INFO:tasks.rados.rados.0.smithi138.stdout:1430: writing smithi138138488-41 from 2894750 to 3352667 tid 3 2023-06-27T12:44:07.660 INFO:tasks.rados.rados.0.smithi138.stdout:1431: snap_remove snap 164 2023-06-27T12:44:07.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:07.985 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:07.980+0000 7f6821782700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:07.988 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:07.984+0000 7f6821782700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:08.032 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:08.034 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:08.035 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:08.511 INFO:tasks.rados.rados.0.smithi138.stdout:1431: done (2 left) 2023-06-27T12:44:08.511 INFO:tasks.rados.rados.0.smithi138.stdout:1432: write oid 1 current snap is 167 2023-06-27T12:44:08.511 INFO:tasks.rados.rados.0.smithi138.stdout:1427: finishing write tid 5 to smithi138138488-4 2023-06-27T12:44:08.511 INFO:tasks.rados.rados.0.smithi138.stdout:1427: oid 4 updating version 0 to 248 2023-06-27T12:44:08.512 INFO:tasks.rados.rados.0.smithi138.stdout:1427: oid 4 updating version 248 to 252 2023-06-27T12:44:08.512 INFO:tasks.rados.rados.0.smithi138.stdout:1427: oid 4 version 252 is already newer than 252 2023-06-27T12:44:08.512 INFO:tasks.rados.rados.0.smithi138.stdout:1427: oid 4 version 252 is already newer than 252 2023-06-27T12:44:08.512 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 4 v 252 (ObjNum 682 snap 167 seq_num 682) dirty exists 2023-06-27T12:44:08.512 INFO:tasks.rados.rados.0.smithi138.stdout:1427: left oid 4 (ObjNum 682 snap 167 seq_num 682) 2023-06-27T12:44:08.512 INFO:tasks.rados.rados.0.smithi138.stdout:1432: seq_num 686 ranges {729602=492584,1843118=684230,3322173=207429} 2023-06-27T12:44:08.517 INFO:tasks.rados.rados.0.smithi138.stdout:1432: writing smithi138138488-1 from 729602 to 1222186 tid 1 2023-06-27T12:44:08.525 INFO:tasks.rados.rados.0.smithi138.stdout:1432: writing smithi138138488-1 from 1843118 to 2527348 tid 2 2023-06-27T12:44:08.531 INFO:tasks.rados.rados.0.smithi138.stdout:1432: writing smithi138138488-1 from 3322173 to 3529602 tid 3 2023-06-27T12:44:08.531 INFO:tasks.rados.rados.0.smithi138.stdout:1427: done (2 left) 2023-06-27T12:44:08.531 INFO:tasks.rados.rados.0.smithi138.stdout:1433: read oid 21 snap -1 2023-06-27T12:44:08.531 INFO:tasks.rados.rados.0.smithi138.stdout:1433: expect deleted 2023-06-27T12:44:08.532 INFO:tasks.rados.rados.0.smithi138.stdout:1434: read oid 2 snap 162 2023-06-27T12:44:08.532 INFO:tasks.rados.rados.0.smithi138.stdout:1434: expect (ObjNum 645 snap 155 seq_num 645) 2023-06-27T12:44:08.532 INFO:tasks.rados.rados.0.smithi138.stdout:1435: read oid 37 snap -1 2023-06-27T12:44:08.532 INFO:tasks.rados.rados.0.smithi138.stdout:1435: expect deleted 2023-06-27T12:44:08.532 INFO:tasks.rados.rados.0.smithi138.stdout:1436: read oid 22 snap 165 2023-06-27T12:44:08.532 INFO:tasks.rados.rados.0.smithi138.stdout:1436: expect deleted 2023-06-27T12:44:08.532 INFO:tasks.rados.rados.0.smithi138.stdout:1437: read oid 44 snap -1 2023-06-27T12:44:08.532 INFO:tasks.rados.rados.0.smithi138.stdout:1437: expect deleted 2023-06-27T12:44:08.533 INFO:tasks.rados.rados.0.smithi138.stdout:1438: write (excl) oid 32 current snap is 167 2023-06-27T12:44:08.533 INFO:tasks.rados.rados.0.smithi138.stdout:1430: finishing write tid 1 to smithi138138488-41 2023-06-27T12:44:08.533 INFO:tasks.rados.rados.0.smithi138.stdout:1438: seq_num 687 ranges {746409=567833,1901600=544241,3065949=480460} 2023-06-27T12:44:08.538 INFO:tasks.rados.rados.0.smithi138.stdout:1438: writing smithi138138488-32 from 746409 to 1314242 tid 1 2023-06-27T12:44:08.544 INFO:tasks.rados.rados.0.smithi138.stdout:1438: writing smithi138138488-32 from 1901600 to 2445841 tid 2 2023-06-27T12:44:08.551 INFO:tasks.rados.rados.0.smithi138.stdout:1438: writing smithi138138488-32 from 3065949 to 3546409 tid 3 2023-06-27T12:44:08.552 INFO:tasks.rados.rados.0.smithi138.stdout:1439: write (excl) oid 47 current snap is 167 2023-06-27T12:44:08.552 INFO:tasks.rados.rados.0.smithi138.stdout:1430: finishing write tid 2 to smithi138138488-41 2023-06-27T12:44:08.552 INFO:tasks.rados.rados.0.smithi138.stdout:1430: finishing write tid 3 to smithi138138488-41 2023-06-27T12:44:08.552 INFO:tasks.rados.rados.0.smithi138.stdout:1430: finishing write tid 4 to smithi138138488-41 2023-06-27T12:44:08.552 INFO:tasks.rados.rados.0.smithi138.stdout:1430: finishing write tid 5 to smithi138138488-41 2023-06-27T12:44:08.552 INFO:tasks.rados.rados.0.smithi138.stdout:1430: oid 41 updating version 0 to 214 2023-06-27T12:44:08.552 INFO:tasks.rados.rados.0.smithi138.stdout:1430: oid 41 version 214 is already newer than 214 2023-06-27T12:44:08.553 INFO:tasks.rados.rados.0.smithi138.stdout:1430: oid 41 version 214 is already newer than 214 2023-06-27T12:44:08.553 INFO:tasks.rados.rados.0.smithi138.stdout:1430: oid 41 version 214 is already newer than 210 2023-06-27T12:44:08.553 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 41 v 214 (ObjNum 685 snap 167 seq_num 685) dirty exists 2023-06-27T12:44:08.553 INFO:tasks.rados.rados.0.smithi138.stdout:1430: left oid 41 (ObjNum 685 snap 167 seq_num 685) 2023-06-27T12:44:08.553 INFO:tasks.rados.rados.0.smithi138.stdout:1439: seq_num 688 ranges {763216=643082,1843729=687899,3260666=302550} 2023-06-27T12:44:08.559 INFO:tasks.rados.rados.0.smithi138.stdout:1439: writing smithi138138488-47 from 763216 to 1406298 tid 1 2023-06-27T12:44:08.566 INFO:tasks.rados.rados.0.smithi138.stdout:1439: writing smithi138138488-47 from 1843729 to 2531628 tid 2 2023-06-27T12:44:08.572 INFO:tasks.rados.rados.0.smithi138.stdout:1439: writing smithi138138488-47 from 3260666 to 3563216 tid 3 2023-06-27T12:44:08.572 INFO:tasks.rados.rados.0.smithi138.stdout:1430: done (8 left) 2023-06-27T12:44:08.572 INFO:tasks.rados.rados.0.smithi138.stdout:1440: write (excl) oid 30 current snap is 167 2023-06-27T12:44:08.572 INFO:tasks.rados.rados.0.smithi138.stdout:1440: seq_num 689 ranges {780023=718331,1902211=547910,3004442=575581} 2023-06-27T12:44:08.581 INFO:tasks.rados.rados.0.smithi138.stdout:1440: writing smithi138138488-30 from 780023 to 1498354 tid 1 2023-06-27T12:44:08.587 INFO:tasks.rados.rados.0.smithi138.stdout:1440: writing smithi138138488-30 from 1902211 to 2450121 tid 2 2023-06-27T12:44:08.594 INFO:tasks.rados.rados.0.smithi138.stdout:1440: writing smithi138138488-30 from 3004442 to 3580023 tid 3 2023-06-27T12:44:08.594 INFO:tasks.rados.rados.0.smithi138.stdout:1441: read oid 17 snap 165 2023-06-27T12:44:08.595 INFO:tasks.rados.rados.0.smithi138.stdout:1441: expect (ObjNum 656 snap 161 seq_num 656) 2023-06-27T12:44:08.595 INFO:tasks.rados.rados.0.smithi138.stdout:1442: snap_create 2023-06-27T12:44:08.595 INFO:tasks.rados.rados.0.smithi138.stdout:1434: expect (ObjNum 645 snap 155 seq_num 645) 2023-06-27T12:44:08.763 INFO:tasks.rados.rados.0.smithi138.stdout:1438: finishing write tid 1 to smithi138138488-32 2023-06-27T12:44:08.764 INFO:tasks.rados.rados.0.smithi138.stdout:1439: finishing write tid 1 to smithi138138488-47 2023-06-27T12:44:08.764 INFO:tasks.rados.rados.0.smithi138.stdout:1432: finishing write tid 1 to smithi138138488-1 2023-06-27T12:44:08.764 INFO:tasks.rados.rados.0.smithi138.stdout:1432: finishing write tid 2 to smithi138138488-1 2023-06-27T12:44:08.764 INFO:tasks.rados.rados.0.smithi138.stdout:1432: finishing write tid 3 to smithi138138488-1 2023-06-27T12:44:08.764 INFO:tasks.rados.rados.0.smithi138.stdout:1432: finishing write tid 4 to smithi138138488-1 2023-06-27T12:44:08.765 INFO:tasks.rados.rados.0.smithi138.stdout:1440: finishing write tid 1 to smithi138138488-30 2023-06-27T12:44:08.765 INFO:tasks.rados.rados.0.smithi138.stdout:1432: finishing write tid 5 to smithi138138488-1 2023-06-27T12:44:08.765 INFO:tasks.rados.rados.0.smithi138.stdout:1432: oid 1 updating version 0 to 401 2023-06-27T12:44:08.765 INFO:tasks.rados.rados.0.smithi138.stdout:1432: oid 1 version 401 is already newer than 401 2023-06-27T12:44:08.765 INFO:tasks.rados.rados.0.smithi138.stdout:1432: oid 1 version 401 is already newer than 401 2023-06-27T12:44:08.765 INFO:tasks.rados.rados.0.smithi138.stdout:1432: oid 1 version 401 is already newer than 401 2023-06-27T12:44:08.765 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 401 (ObjNum 686 snap 167 seq_num 686) dirty exists 2023-06-27T12:44:08.766 INFO:tasks.rados.rados.0.smithi138.stdout:1432: left oid 1 (ObjNum 686 snap 167 seq_num 686) 2023-06-27T12:44:08.766 INFO:tasks.rados.rados.0.smithi138.stdout:1438: finishing write tid 2 to smithi138138488-32 2023-06-27T12:44:08.766 INFO:tasks.rados.rados.0.smithi138.stdout:1438: finishing write tid 3 to smithi138138488-32 2023-06-27T12:44:08.766 INFO:tasks.rados.rados.0.smithi138.stdout:1438: finishing write tid 4 to smithi138138488-32 2023-06-27T12:44:08.766 INFO:tasks.rados.rados.0.smithi138.stdout:1438: finishing write tid 5 to smithi138138488-32 2023-06-27T12:44:08.766 INFO:tasks.rados.rados.0.smithi138.stdout:1438: oid 32 updating version 0 to 447 2023-06-27T12:44:08.766 INFO:tasks.rados.rados.0.smithi138.stdout:1438: oid 32 version 447 is already newer than 443 2023-06-27T12:44:08.766 INFO:tasks.rados.rados.0.smithi138.stdout:1438: oid 32 version 447 is already newer than 447 2023-06-27T12:44:08.767 INFO:tasks.rados.rados.0.smithi138.stdout:1438: oid 32 version 447 is already newer than 447 2023-06-27T12:44:08.767 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 447 (ObjNum 687 snap 167 seq_num 687) dirty exists 2023-06-27T12:44:08.767 INFO:tasks.rados.rados.0.smithi138.stdout:1438: left oid 32 (ObjNum 687 snap 167 seq_num 687) 2023-06-27T12:44:08.767 INFO:tasks.rados.rados.0.smithi138.stdout:1439: finishing write tid 2 to smithi138138488-47 2023-06-27T12:44:08.767 INFO:tasks.rados.rados.0.smithi138.stdout:1439: finishing write tid 3 to smithi138138488-47 2023-06-27T12:44:08.767 INFO:tasks.rados.rados.0.smithi138.stdout:1439: finishing write tid 4 to smithi138138488-47 2023-06-27T12:44:08.767 INFO:tasks.rados.rados.0.smithi138.stdout:1439: finishing write tid 5 to smithi138138488-47 2023-06-27T12:44:08.767 INFO:tasks.rados.rados.0.smithi138.stdout:1439: oid 47 updating version 0 to 207 2023-06-27T12:44:08.768 INFO:tasks.rados.rados.0.smithi138.stdout:1439: oid 47 version 207 is already newer than 207 2023-06-27T12:44:08.768 INFO:tasks.rados.rados.0.smithi138.stdout:1439: oid 47 version 207 is already newer than 204 2023-06-27T12:44:08.768 INFO:tasks.rados.rados.0.smithi138.stdout:1439: oid 47 version 207 is already newer than 207 2023-06-27T12:44:08.768 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 47 v 207 (ObjNum 688 snap 167 seq_num 688) dirty exists 2023-06-27T12:44:08.768 INFO:tasks.rados.rados.0.smithi138.stdout:1439: left oid 47 (ObjNum 688 snap 167 seq_num 688) 2023-06-27T12:44:08.768 INFO:tasks.rados.rados.0.smithi138.stdout:1440: finishing write tid 2 to smithi138138488-30 2023-06-27T12:44:08.768 INFO:tasks.rados.rados.0.smithi138.stdout:1440: finishing write tid 3 to smithi138138488-30 2023-06-27T12:44:08.768 INFO:tasks.rados.rados.0.smithi138.stdout:1440: finishing write tid 4 to smithi138138488-30 2023-06-27T12:44:08.769 INFO:tasks.rados.rados.0.smithi138.stdout:1440: finishing write tid 5 to smithi138138488-30 2023-06-27T12:44:08.769 INFO:tasks.rados.rados.0.smithi138.stdout:1440: oid 30 updating version 0 to 257 2023-06-27T12:44:08.769 INFO:tasks.rados.rados.0.smithi138.stdout:1440: oid 30 version 257 is already newer than 257 2023-06-27T12:44:08.769 INFO:tasks.rados.rados.0.smithi138.stdout:1440: oid 30 version 257 is already newer than 257 2023-06-27T12:44:08.769 INFO:tasks.rados.rados.0.smithi138.stdout:1440: oid 30 version 257 is already newer than 253 2023-06-27T12:44:08.769 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 257 (ObjNum 689 snap 167 seq_num 689) dirty exists 2023-06-27T12:44:08.769 INFO:tasks.rados.rados.0.smithi138.stdout:1440: left oid 30 (ObjNum 689 snap 167 seq_num 689) 2023-06-27T12:44:08.770 INFO:tasks.rados.rados.0.smithi138.stdout:1441: expect (ObjNum 656 snap 161 seq_num 656) 2023-06-27T12:44:09.511 INFO:tasks.rados.rados.0.smithi138.stdout:1432: done (10 left) 2023-06-27T12:44:09.511 INFO:tasks.rados.rados.0.smithi138.stdout:1433: done (9 left) 2023-06-27T12:44:09.511 INFO:tasks.rados.rados.0.smithi138.stdout:1434: done (8 left) 2023-06-27T12:44:09.511 INFO:tasks.rados.rados.0.smithi138.stdout:1435: done (7 left) 2023-06-27T12:44:09.512 INFO:tasks.rados.rados.0.smithi138.stdout:1436: done (6 left) 2023-06-27T12:44:09.512 INFO:tasks.rados.rados.0.smithi138.stdout:1437: done (5 left) 2023-06-27T12:44:09.512 INFO:tasks.rados.rados.0.smithi138.stdout:1438: done (4 left) 2023-06-27T12:44:09.512 INFO:tasks.rados.rados.0.smithi138.stdout:1439: done (3 left) 2023-06-27T12:44:09.512 INFO:tasks.rados.rados.0.smithi138.stdout:1440: done (2 left) 2023-06-27T12:44:09.512 INFO:tasks.rados.rados.0.smithi138.stdout:1441: done (1 left) 2023-06-27T12:44:09.512 INFO:tasks.rados.rados.0.smithi138.stdout:1442: done (0 left) 2023-06-27T12:44:09.512 INFO:tasks.rados.rados.0.smithi138.stdout:1443: delete oid 47 current snap is 168 2023-06-27T12:44:09.580 INFO:tasks.rados.rados.0.smithi138.stdout:1443: done (0 left) 2023-06-27T12:44:09.580 INFO:tasks.rados.rados.0.smithi138.stdout:1444: write oid 27 current snap is 168 2023-06-27T12:44:09.580 INFO:tasks.rados.rados.0.smithi138.stdout:1444: seq_num 691 ranges {413637=468829,1619175=435226,2542935=758255} 2023-06-27T12:44:09.585 INFO:tasks.rados.rados.0.smithi138.stdout:1444: writing smithi138138488-27 from 413637 to 882466 tid 1 2023-06-27T12:44:09.591 INFO:tasks.rados.rados.0.smithi138.stdout:1444: writing smithi138138488-27 from 1619175 to 2054401 tid 2 2023-06-27T12:44:09.598 INFO:tasks.rados.rados.0.smithi138.stdout:1444: writing smithi138138488-27 from 2542935 to 3301190 tid 3 2023-06-27T12:44:09.598 INFO:tasks.rados.rados.0.smithi138.stdout:1445: delete oid 38 current snap is 168 2023-06-27T12:44:09.601 INFO:tasks.rados.rados.0.smithi138.stdout:1445: done (1 left) 2023-06-27T12:44:09.602 INFO:tasks.rados.rados.0.smithi138.stdout:1446: snap_remove snap 160 2023-06-27T12:44:09.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~215s 2023-06-27T12:44:10.036 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:10.162 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:10.157+0000 7f7d14226700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:10.165 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:10.161+0000 7f7d14226700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:10.211 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:10.213 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:10.214 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:10.459 INFO:tasks.rados.rados.0.smithi138.stdout:1446: done (1 left) 2023-06-27T12:44:10.459 INFO:tasks.rados.rados.0.smithi138.stdout:1447: write (excl) oid 42 current snap is 168 2023-06-27T12:44:10.459 INFO:tasks.rados.rados.0.smithi138.stdout:1444: finishing write tid 1 to smithi138138488-27 2023-06-27T12:44:10.459 INFO:tasks.rados.rados.0.smithi138.stdout:1447: seq_num 693 ranges {447251=735680,1736139=438895,2597781=780799} 2023-06-27T12:44:10.466 INFO:tasks.rados.rados.0.smithi138.stdout:1447: writing smithi138138488-42 from 447251 to 1182931 tid 1 2023-06-27T12:44:10.472 INFO:tasks.rados.rados.0.smithi138.stdout:1447: writing smithi138138488-42 from 1736139 to 2175034 tid 2 2023-06-27T12:44:10.479 INFO:tasks.rados.rados.0.smithi138.stdout:1447: writing smithi138138488-42 from 2597781 to 3378580 tid 3 2023-06-27T12:44:10.479 INFO:tasks.rados.rados.0.smithi138.stdout:1448: snap_remove snap 162 2023-06-27T12:44:11.465 INFO:tasks.rados.rados.0.smithi138.stdout:1448: done (2 left) 2023-06-27T12:44:11.465 INFO:tasks.rados.rados.0.smithi138.stdout:1449: write (excl) oid 44 current snap is 168 2023-06-27T12:44:11.465 INFO:tasks.rados.rados.0.smithi138.stdout:1444: finishing write tid 2 to smithi138138488-27 2023-06-27T12:44:11.466 INFO:tasks.rados.rados.0.smithi138.stdout:1449: seq_num 694 ranges {464058=410929,1394621=698906,2625204=592071,3656107=7951} 2023-06-27T12:44:11.470 INFO:tasks.rados.rados.0.smithi138.stdout:1449: writing smithi138138488-44 from 464058 to 874987 tid 1 2023-06-27T12:44:11.477 INFO:tasks.rados.rados.0.smithi138.stdout:1449: writing smithi138138488-44 from 1394621 to 2093527 tid 2 2023-06-27T12:44:11.483 INFO:tasks.rados.rados.0.smithi138.stdout:1449: writing smithi138138488-44 from 2625204 to 3217275 tid 3 2023-06-27T12:44:11.486 INFO:tasks.rados.rados.0.smithi138.stdout:1449: writing smithi138138488-44 from 3656107 to 3664058 tid 4 2023-06-27T12:44:11.486 INFO:tasks.rados.rados.0.smithi138.stdout:1450: snap_create 2023-06-27T12:44:11.486 INFO:tasks.rados.rados.0.smithi138.stdout:1444: finishing write tid 3 to smithi138138488-27 2023-06-27T12:44:11.486 INFO:tasks.rados.rados.0.smithi138.stdout:1444: finishing write tid 4 to smithi138138488-27 2023-06-27T12:44:11.487 INFO:tasks.rados.rados.0.smithi138.stdout:1444: finishing write tid 5 to smithi138138488-27 2023-06-27T12:44:11.487 INFO:tasks.rados.rados.0.smithi138.stdout:1444: oid 27 updating version 0 to 214 2023-06-27T12:44:11.487 INFO:tasks.rados.rados.0.smithi138.stdout:1444: oid 27 version 214 is already newer than 214 2023-06-27T12:44:11.487 INFO:tasks.rados.rados.0.smithi138.stdout:1444: oid 27 version 214 is already newer than 212 2023-06-27T12:44:11.487 INFO:tasks.rados.rados.0.smithi138.stdout:1444: oid 27 version 214 is already newer than 214 2023-06-27T12:44:11.487 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 214 (ObjNum 691 snap 168 seq_num 691) dirty exists 2023-06-27T12:44:11.487 INFO:tasks.rados.rados.0.smithi138.stdout:1444: left oid 27 (ObjNum 691 snap 168 seq_num 691) 2023-06-27T12:44:11.487 INFO:tasks.rados.rados.0.smithi138.stdout:1447: finishing write tid 1 to smithi138138488-42 2023-06-27T12:44:11.488 INFO:tasks.rados.rados.0.smithi138.stdout:1447: finishing write tid 2 to smithi138138488-42 2023-06-27T12:44:11.488 INFO:tasks.rados.rados.0.smithi138.stdout:1447: finishing write tid 3 to smithi138138488-42 2023-06-27T12:44:11.488 INFO:tasks.rados.rados.0.smithi138.stdout:1447: finishing write tid 4 to smithi138138488-42 2023-06-27T12:44:11.488 INFO:tasks.rados.rados.0.smithi138.stdout:1447: finishing write tid 5 to smithi138138488-42 2023-06-27T12:44:11.488 INFO:tasks.rados.rados.0.smithi138.stdout:1447: oid 42 updating version 0 to 188 2023-06-27T12:44:11.488 INFO:tasks.rados.rados.0.smithi138.stdout:1447: oid 42 version 188 is already newer than 184 2023-06-27T12:44:11.488 INFO:tasks.rados.rados.0.smithi138.stdout:1447: oid 42 version 188 is already newer than 188 2023-06-27T12:44:11.488 INFO:tasks.rados.rados.0.smithi138.stdout:1447: oid 42 version 188 is already newer than 188 2023-06-27T12:44:11.489 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 188 (ObjNum 693 snap 168 seq_num 693) dirty exists 2023-06-27T12:44:11.489 INFO:tasks.rados.rados.0.smithi138.stdout:1447: left oid 42 (ObjNum 693 snap 168 seq_num 693) 2023-06-27T12:44:11.553 INFO:tasks.rados.rados.0.smithi138.stdout:1449: finishing write tid 1 to smithi138138488-44 2023-06-27T12:44:11.582 INFO:tasks.rados.rados.0.smithi138.stdout:1449: finishing write tid 2 to smithi138138488-44 2023-06-27T12:44:11.584 INFO:tasks.rados.rados.0.smithi138.stdout:1449: finishing write tid 3 to smithi138138488-44 2023-06-27T12:44:11.585 INFO:tasks.rados.rados.0.smithi138.stdout:1449: finishing write tid 4 to smithi138138488-44 2023-06-27T12:44:11.586 INFO:tasks.rados.rados.0.smithi138.stdout:1449: finishing write tid 5 to smithi138138488-44 2023-06-27T12:44:11.587 INFO:tasks.rados.rados.0.smithi138.stdout:1449: finishing write tid 6 to smithi138138488-44 2023-06-27T12:44:11.587 INFO:tasks.rados.rados.0.smithi138.stdout:1449: oid 44 updating version 0 to 415 2023-06-27T12:44:11.587 INFO:tasks.rados.rados.0.smithi138.stdout:1449: oid 44 version 415 is already newer than 415 2023-06-27T12:44:11.587 INFO:tasks.rados.rados.0.smithi138.stdout:1449: oid 44 version 415 is already newer than 415 2023-06-27T12:44:11.587 INFO:tasks.rados.rados.0.smithi138.stdout:1449: oid 44 version 415 is already newer than 415 2023-06-27T12:44:11.587 INFO:tasks.rados.rados.0.smithi138.stdout:1449: oid 44 version 415 is already newer than 411 2023-06-27T12:44:11.587 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 415 (ObjNum 694 snap 168 seq_num 694) dirty exists 2023-06-27T12:44:11.588 INFO:tasks.rados.rados.0.smithi138.stdout:1449: left oid 44 (ObjNum 694 snap 168 seq_num 694) 2023-06-27T12:44:12.215 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:12.342 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:12.337+0000 7f5575114700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:12.345 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:12.341+0000 7f5575114700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:12.390 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:12.392 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:12.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:12.469 INFO:tasks.rados.rados.0.smithi138.stdout:1444: done (3 left) 2023-06-27T12:44:12.469 INFO:tasks.rados.rados.0.smithi138.stdout:1447: done (2 left) 2023-06-27T12:44:12.469 INFO:tasks.rados.rados.0.smithi138.stdout:1449: done (1 left) 2023-06-27T12:44:12.469 INFO:tasks.rados.rados.0.smithi138.stdout:1450: done (0 left) 2023-06-27T12:44:12.469 INFO:tasks.rados.rados.0.smithi138.stdout:1451: read oid 26 snap 165 2023-06-27T12:44:12.470 INFO:tasks.rados.rados.0.smithi138.stdout:1451: expect (ObjNum 612 snap 145 seq_num 612) 2023-06-27T12:44:12.470 INFO:tasks.rados.rados.0.smithi138.stdout:1452: write (excl) oid 44 current snap is 169 2023-06-27T12:44:12.470 INFO:tasks.rados.rados.0.smithi138.stdout:1452: seq_num 695 ranges {480865=486178,1453103=442564,2652627=519696} 2023-06-27T12:44:12.474 INFO:tasks.rados.rados.0.smithi138.stdout:1452: writing smithi138138488-44 from 480865 to 967043 tid 1 2023-06-27T12:44:12.479 INFO:tasks.rados.rados.0.smithi138.stdout:1452: writing smithi138138488-44 from 1453103 to 1895667 tid 2 2023-06-27T12:44:12.486 INFO:tasks.rados.rados.0.smithi138.stdout:1452: writing smithi138138488-44 from 2652627 to 3172323 tid 3 2023-06-27T12:44:12.486 INFO:tasks.rados.rados.0.smithi138.stdout:1453: write oid 20 current snap is 169 2023-06-27T12:44:12.486 INFO:tasks.rados.rados.0.smithi138.stdout:1453: seq_num 696 ranges {497672=561427,1795232=702575,2963697=730968} 2023-06-27T12:44:12.492 INFO:tasks.rados.rados.0.smithi138.stdout:1453: writing smithi138138488-20 from 497672 to 1059099 tid 1 2023-06-27T12:44:12.493 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:12.493 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:12.494 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:12.503 INFO:tasks.rados.rados.0.smithi138.stdout:1453: writing smithi138138488-20 from 1795232 to 2497807 tid 2 2023-06-27T12:44:12.510 INFO:tasks.rados.rados.0.smithi138.stdout:1453: writing smithi138138488-20 from 2963697 to 3694665 tid 3 2023-06-27T12:44:12.510 INFO:tasks.rados.rados.0.smithi138.stdout:1454: delete oid 16 current snap is 169 2023-06-27T12:44:12.514 INFO:tasks.rados.rados.0.smithi138.stdout:1451: expect (ObjNum 612 snap 145 seq_num 612) 2023-06-27T12:44:12.620 INFO:tasks.rados.rados.0.smithi138.stdout:1452: finishing write tid 1 to smithi138138488-44 2023-06-27T12:44:12.621 INFO:tasks.rados.rados.0.smithi138.stdout:1453: finishing write tid 1 to smithi138138488-20 2023-06-27T12:44:12.621 INFO:tasks.rados.rados.0.smithi138.stdout:1453: finishing write tid 2 to smithi138138488-20 2023-06-27T12:44:12.621 INFO:tasks.rados.rados.0.smithi138.stdout:1453: finishing write tid 3 to smithi138138488-20 2023-06-27T12:44:12.622 INFO:tasks.rados.rados.0.smithi138.stdout:1453: finishing write tid 4 to smithi138138488-20 2023-06-27T12:44:12.622 INFO:tasks.rados.rados.0.smithi138.stdout:1453: finishing write tid 5 to smithi138138488-20 2023-06-27T12:44:12.622 INFO:tasks.rados.rados.0.smithi138.stdout:1453: oid 20 updating version 0 to 223 2023-06-27T12:44:12.622 INFO:tasks.rados.rados.0.smithi138.stdout:1453: oid 20 version 223 is already newer than 222 2023-06-27T12:44:12.622 INFO:tasks.rados.rados.0.smithi138.stdout:1453: oid 20 version 223 is already newer than 223 2023-06-27T12:44:12.623 INFO:tasks.rados.rados.0.smithi138.stdout:1453: oid 20 version 223 is already newer than 223 2023-06-27T12:44:12.623 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 223 (ObjNum 696 snap 169 seq_num 696) dirty exists 2023-06-27T12:44:12.623 INFO:tasks.rados.rados.0.smithi138.stdout:1453: left oid 20 (ObjNum 696 snap 169 seq_num 696) 2023-06-27T12:44:12.645 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:12.638+0000 7fce499df700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:12.648 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:12.645+0000 7fce499df700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:12.649 INFO:tasks.rados.rados.0.smithi138.stdout:1452: finishing write tid 2 to smithi138138488-44 2023-06-27T12:44:12.651 INFO:tasks.rados.rados.0.smithi138.stdout:1452: finishing write tid 3 to smithi138138488-44 2023-06-27T12:44:12.658 INFO:tasks.rados.rados.0.smithi138.stdout:1452: finishing write tid 4 to smithi138138488-44 2023-06-27T12:44:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1452: finishing write tid 5 to smithi138138488-44 2023-06-27T12:44:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1452: oid 44 updating version 0 to 420 2023-06-27T12:44:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1452: oid 44 version 420 is already newer than 420 2023-06-27T12:44:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1452: oid 44 version 420 is already newer than 416 2023-06-27T12:44:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1452: oid 44 version 420 is already newer than 420 2023-06-27T12:44:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 420 (ObjNum 695 snap 169 seq_num 695) dirty exists 2023-06-27T12:44:12.661 INFO:tasks.rados.rados.0.smithi138.stdout:1452: left oid 44 (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:44:12.662 INFO:tasks.rados.rados.0.smithi138.stdout:1451: done (3 left) 2023-06-27T12:44:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1452: done (2 left) 2023-06-27T12:44:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1453: done (1 left) 2023-06-27T12:44:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1454: done (0 left) 2023-06-27T12:44:12.663 INFO:tasks.rados.rados.0.smithi138.stdout:1455: snap_create 2023-06-27T12:44:12.847 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:12.849 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:12.861 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":540,"stamp":"2023-06-27T12:44:12.463602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":90526363,"num_objects":361,"num_object_clones":314,"num_object_copies":1083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":2100,"num_read_kb":1546245,"num_write":1893,"num_write_kb":551985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3463,"ondisk_log_size":3463,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1053404,"kb_used_data":917436,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376433956,"statfs":{"total":386547056640,"available":385468370944,"internally_reserved":0,"allocated":939454464,"data_stored":1079008539,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-218491,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":29,"num_read_kb":17882,"num_write":54,"num_write_kb":16720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.960231"},"pg_stats":[{"pgid":"3.f","version":"344'39","reported_seq":841,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.563314+0000","last_change":"2023-06-27T12:44:08.566207+0000","last_active":"2023-06-27T12:44:08.563314+0000","last_peered":"2023-06-27T12:44:08.563314+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:08.563314+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":11,"num_read_kb":6,"num_write":28,"num_write_kb":7205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"354'207","reported_seq":1104,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.744678+0000","last_change":"2023-06-27T12:44:08.563709+0000","last_active":"2023-06-27T12:44:08.744678+0000","last_peered":"2023-06-27T12:44:08.744678+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:08.744678+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12437675,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":199077,"num_write":147,"num_write_kb":43692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"349'273","reported_seq":1086,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.516215+0000","last_change":"2023-06-27T12:44:08.705720+0000","last_active":"2023-06-27T12:44:08.516215+0000","last_peered":"2023-06-27T12:44:08.516215+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:08.516215+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8689549,"num_objects":31,"num_object_clones":26,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":237,"num_read_kb":160924,"num_write":174,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":31}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"352'179","reported_seq":1038,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.586628+0000","last_change":"2023-06-27T12:44:08.665408+0000","last_active":"2023-06-27T12:44:08.586628+0000","last_peered":"2023-06-27T12:44:08.586628+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:08.586628+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462704,"num_objects":27,"num_object_clones":23,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":164,"num_read_kb":111757,"num_write":124,"num_write_kb":35247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":27}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":589,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.562884+0000","last_change":"2023-06-27T12:44:08.565049+0000","last_active":"2023-06-27T12:44:08.562884+0000","last_peered":"2023-06-27T12:44:08.562884+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:08.562884+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"334'408","reported_seq":1114,"reported_epoch":354,"state":"active+clean","last_fresh":"2023-06-27T12:44:08.561924+0000","last_change":"2023-06-27T12:44:08.564296+0000","last_active":"2023-06-27T12:44:08.561924+0000","last_peered":"2023-06-27T12:44:08.561924+0000","last_clean":"2023-06-27T12:44:08.561924+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:08.561924+0000","last_undegraded":"2023-06-27T12:44:08.561924+0000","last_fullsized":"2023-06-27T12:44:08.561924+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2437147,"num_objects":45,"num_object_clones":39,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":5,"num_read":309,"num_read_kb":193213,"num_write":232,"num_write_kb":66787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"352'203","reported_seq":935,"reported_epoch":354,"state":"active+clean+snaptrim","last_fresh":"2023-06-27T12:44:08.571069+0000","last_change":"2023-06-27T12:44:08.571069+0000","last_active":"2023-06-27T12:44:08.571069+0000","last_peered":"2023-06-27T12:44:08.571069+0000","last_clean":"2023-06-27T12:44:08.571069+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:08.571069+0000","last_undegraded":"2023-06-27T12:44:08.571069+0000","last_fullsized":"2023-06-27T12:44:08.571069+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":132,"num_read_kb":85550,"num_write":127,"num_write_kb":37347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":877,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.511164+0000","last_change":"2023-06-27T12:44:08.518507+0000","last_active":"2023-06-27T12:44:08.511164+0000","last_peered":"2023-06-27T12:44:08.511164+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:08.511164+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":594,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-06-27T12:44:08.548773+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:08.548773+0000","last_peered":"2023-06-27T12:44:08.548773+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:08.548773+0000","last_undegraded":"2023-06-27T12:44:08.548773+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":785,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-06-27T12:44:08.519431+0000","last_change":"2023-06-27T12:44:08.525646+0000","last_active":"2023-06-27T12:44:08.519431+0000","last_peered":"2023-06-27T12:44:08.519431+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:08.519431+0000","last_undegraded":"2023-06-27T12:44:08.519431+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":575,"reported_epoch":354,"state":"active+clean","last_fresh":"2023-06-27T12:44:08.549050+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:08.549050+0000","last_peered":"2023-06-27T12:44:08.549050+0000","last_clean":"2023-06-27T12:44:08.549050+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:08.549050+0000","last_undegraded":"2023-06-27T12:44:08.549050+0000","last_fullsized":"2023-06-27T12:44:08.549050+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"349'194","reported_seq":1015,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.548300+0000","last_change":"2023-06-27T12:44:08.607435+0000","last_active":"2023-06-27T12:44:08.548300+0000","last_peered":"2023-06-27T12:44:08.548300+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:08.548300+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6101205,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":80,"num_read_kb":70438,"num_write":124,"num_write_kb":34515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":556,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-06-27T12:44:08.517236+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:08.517236+0000","last_peered":"2023-06-27T12:44:08.517236+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:08.517236+0000","last_undegraded":"2023-06-27T12:44:08.517236+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"354'447","reported_seq":696,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.654354+0000","last_change":"2023-06-27T12:44:08.603804+0000","last_active":"2023-06-27T12:44:08.654354+0000","last_peered":"2023-06-27T12:44:08.654354+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:08.654354+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5865907,"num_objects":26,"num_object_clones":24,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":41842,"num_write":131,"num_write_kb":40677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":553,"reported_epoch":354,"state":"active+clean","last_fresh":"2023-06-27T12:44:08.520415+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:08.520415+0000","last_peered":"2023-06-27T12:44:08.520415+0000","last_clean":"2023-06-27T12:44:08.520415+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:08.520415+0000","last_undegraded":"2023-06-27T12:44:08.520415+0000","last_fullsized":"2023-06-27T12:44:08.520415+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":327,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-06-27T12:44:08.561646+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:08.561646+0000","last_peered":"2023-06-27T12:44:08.561646+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:08.561646+0000","last_undegraded":"2023-06-27T12:44:08.561646+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"354'401","reported_seq":1226,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.711335+0000","last_change":"2023-06-27T12:44:08.713734+0000","last_active":"2023-06-27T12:44:08.711335+0000","last_peered":"2023-06-27T12:44:08.711335+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:08.711335+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790292,"num_objects":51,"num_object_clones":45,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":312,"num_read_kb":189209,"num_write":283,"num_write_kb":83218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":575,"reported_epoch":354,"state":"active+clean","last_fresh":"2023-06-27T12:44:08.556482+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:08.556482+0000","last_peered":"2023-06-27T12:44:08.556482+0000","last_clean":"2023-06-27T12:44:08.556482+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:08.556482+0000","last_undegraded":"2023-06-27T12:44:08.556482+0000","last_fullsized":"2023-06-27T12:44:08.556482+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"354'257","reported_seq":621,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.738966+0000","last_change":"2023-06-27T12:44:08.542611+0000","last_active":"2023-06-27T12:44:08.738966+0000","last_peered":"2023-06-27T12:44:08.738966+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:08.738966+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9866138,"num_objects":19,"num_object_clones":19,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59443,"num_write":105,"num_write_kb":30476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":337,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-06-27T12:44:08.540060+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:08.540060+0000","last_peered":"2023-06-27T12:44:08.540060+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:08.540060+0000","last_undegraded":"2023-06-27T12:44:08.540060+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"353'214","reported_seq":1058,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.550021+0000","last_change":"2023-06-27T12:44:08.550473+0000","last_active":"2023-06-27T12:44:08.550021+0000","last_peered":"2023-06-27T12:44:08.550021+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:08.550021+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8588773,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":112,"num_read_kb":121605,"num_write":141,"num_write_kb":40461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"352'135","reported_seq":572,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.511956+0000","last_change":"2023-06-27T12:44:08.696088+0000","last_active":"2023-06-27T12:44:08.511956+0000","last_peered":"2023-06-27T12:44:08.511956+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:08.511956+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2252270,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":99,"num_read_kb":56966,"num_write":53,"num_write_kb":14834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":594,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-06-27T12:44:08.538998+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:08.538998+0000","last_peered":"2023-06-27T12:44:08.538998+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:08.538998+0000","last_undegraded":"2023-06-27T12:44:08.538998+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":553,"reported_epoch":354,"state":"active+clean","last_fresh":"2023-06-27T12:44:08.517327+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:08.517327+0000","last_peered":"2023-06-27T12:44:08.517327+0000","last_clean":"2023-06-27T12:44:08.517327+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:08.517327+0000","last_undegraded":"2023-06-27T12:44:08.517327+0000","last_fullsized":"2023-06-27T12:44:08.517327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"345'226","reported_seq":1053,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:08.543355+0000","last_change":"2023-06-27T12:44:08.547942+0000","last_active":"2023-06-27T12:44:08.543355+0000","last_peered":"2023-06-27T12:44:08.543355+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:08.543355+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117814,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":121,"num_read_kb":132245,"num_write":120,"num_write_kb":35273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":90526363,"num_objects":361,"num_object_clones":314,"num_object_copies":1083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":2100,"num_read_kb":1546245,"num_write":1893,"num_write_kb":551985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3366,"ondisk_log_size":3366,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673335,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":275476,"kb_used_data":240460,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94096364,"statfs":{"total":96636764160,"available":96354676736,"internally_reserved":0,"allocated":246231040,"data_stored":284086412,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706040,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":286836,"kb_used_data":251820,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94085004,"statfs":{"total":96636764160,"available":96343044096,"internally_reserved":0,"allocated":257863680,"data_stored":293772619,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673335,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":309812,"kb_used_data":274796,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94062028,"statfs":{"total":96636764160,"available":96319516672,"internally_reserved":0,"allocated":281391104,"data_stored":324109956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:13.528 INFO:tasks.rados.rados.0.smithi138.stdout:1455: done (0 left) 2023-06-27T12:44:13.528 INFO:tasks.rados.rados.0.smithi138.stdout:1456: snap_remove snap 168 2023-06-27T12:44:14.394 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:1456: done (0 left) 2023-06-27T12:44:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:1457: read oid 36 snap -1 2023-06-27T12:44:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:1457: expect (ObjNum 668 snap 165 seq_num 668) 2023-06-27T12:44:14.533 INFO:tasks.rados.rados.0.smithi138.stdout:1458: snap_create 2023-06-27T12:44:14.534 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:14.528+0000 7ff978331700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:14.536 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:14.532+0000 7ff978331700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:14.586 INFO:tasks.rados.rados.0.smithi138.stdout:1457: expect (ObjNum 668 snap 165 seq_num 668) 2023-06-27T12:44:14.607 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:14.611 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:14.612 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:15.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~220s 2023-06-27T12:44:15.534 INFO:tasks.rados.rados.0.smithi138.stdout:1457: done (1 left) 2023-06-27T12:44:15.534 INFO:tasks.rados.rados.0.smithi138.stdout:1458: done (0 left) 2023-06-27T12:44:15.534 INFO:tasks.rados.rados.0.smithi138.stdout:1459: write (excl) oid 38 current snap is 171 2023-06-27T12:44:15.534 INFO:tasks.rados.rados.0.smithi138.stdout:1459: seq_num 698 ranges {531286=711925,1912196=706244,3018543=712743} 2023-06-27T12:44:15.543 INFO:tasks.rados.rados.0.smithi138.stdout:1459: writing smithi138138488-38 from 531286 to 1243211 tid 1 2023-06-27T12:44:15.552 INFO:tasks.rados.rados.0.smithi138.stdout:1459: writing smithi138138488-38 from 1912196 to 2618440 tid 2 2023-06-27T12:44:15.561 INFO:tasks.rados.rados.0.smithi138.stdout:1459: writing smithi138138488-38 from 3018543 to 3731286 tid 3 2023-06-27T12:44:15.561 INFO:tasks.rados.rados.0.smithi138.stdout:1460: delete oid 12 current snap is 171 2023-06-27T12:44:15.630 INFO:tasks.rados.rados.0.smithi138.stdout:1460: done (1 left) 2023-06-27T12:44:15.630 INFO:tasks.rados.rados.0.smithi138.stdout:1461: write oid 36 current snap is 171 2023-06-27T12:44:15.630 INFO:tasks.rados.rados.0.smithi138.stdout:1461: seq_num 700 ranges {564900=578776,1629160=709913,3073389=492409} 2023-06-27T12:44:15.636 INFO:tasks.rados.rados.0.smithi138.stdout:1461: writing smithi138138488-36 from 564900 to 1143676 tid 1 2023-06-27T12:44:15.643 INFO:tasks.rados.rados.0.smithi138.stdout:1461: writing smithi138138488-36 from 1629160 to 2339073 tid 2 2023-06-27T12:44:15.651 INFO:tasks.rados.rados.0.smithi138.stdout:1461: writing smithi138138488-36 from 3073389 to 3565798 tid 3 2023-06-27T12:44:15.652 INFO:tasks.rados.rados.0.smithi138.stdout:1462: read oid 17 snap -1 2023-06-27T12:44:15.652 INFO:tasks.rados.rados.0.smithi138.stdout:1462: expect (ObjNum 656 snap 161 seq_num 656) 2023-06-27T12:44:15.652 INFO:tasks.rados.rados.0.smithi138.stdout:1463: read oid 40 snap 170 2023-06-27T12:44:15.652 INFO:tasks.rados.rados.0.smithi138.stdout:1463: expect (ObjNum 679 snap 167 seq_num 679) 2023-06-27T12:44:15.652 INFO:tasks.rados.rados.0.smithi138.stdout:1464: read oid 16 snap 165 2023-06-27T12:44:15.652 INFO:tasks.rados.rados.0.smithi138.stdout:1464: expect deleted 2023-06-27T12:44:15.652 INFO:tasks.rados.rados.0.smithi138.stdout:1465: write oid 27 current snap is 171 2023-06-27T12:44:15.653 INFO:tasks.rados.rados.0.smithi138.stdout:1459: finishing write tid 1 to smithi138138488-38 2023-06-27T12:44:15.653 INFO:tasks.rados.rados.0.smithi138.stdout:1465: seq_num 701 ranges {581707=654025,1687642=453571,2700812=703681} 2023-06-27T12:44:15.658 INFO:tasks.rados.rados.0.smithi138.stdout:1465: writing smithi138138488-27 from 581707 to 1235732 tid 1 2023-06-27T12:44:15.663 INFO:tasks.rados.rados.0.smithi138.stdout:1465: writing smithi138138488-27 from 1687642 to 2141213 tid 2 2023-06-27T12:44:15.671 INFO:tasks.rados.rados.0.smithi138.stdout:1465: writing smithi138138488-27 from 2700812 to 3404493 tid 3 2023-06-27T12:44:15.671 INFO:tasks.rados.rados.0.smithi138.stdout:1466: write oid 22 current snap is 171 2023-06-27T12:44:15.671 INFO:tasks.rados.rados.0.smithi138.stdout:1459: finishing write tid 2 to smithi138138488-38 2023-06-27T12:44:15.671 INFO:tasks.rados.rados.0.smithi138.stdout:1466: seq_num 702 ranges {598514=729274,1746124=597229,3128235=514953} 2023-06-27T12:44:15.678 INFO:tasks.rados.rados.0.smithi138.stdout:1466: writing smithi138138488-22 from 598514 to 1327788 tid 1 2023-06-27T12:44:15.684 INFO:tasks.rados.rados.0.smithi138.stdout:1466: writing smithi138138488-22 from 1746124 to 2343353 tid 2 2023-06-27T12:44:15.692 INFO:tasks.rados.rados.0.smithi138.stdout:1466: writing smithi138138488-22 from 3128235 to 3643188 tid 3 2023-06-27T12:44:15.692 INFO:tasks.rados.rados.0.smithi138.stdout:1467: read oid 9 snap -1 2023-06-27T12:44:15.692 INFO:tasks.rados.rados.0.smithi138.stdout:1467: expect (ObjNum 670 snap 165 seq_num 670) 2023-06-27T12:44:15.692 INFO:tasks.rados.rados.0.smithi138.stdout:1468: write (excl) oid 29 current snap is 171 2023-06-27T12:44:15.693 INFO:tasks.rados.rados.0.smithi138.stdout:1468: seq_num 703 ranges {615321=404523,1804606=457240,2755658=726225} 2023-06-27T12:44:15.699 INFO:tasks.rados.rados.0.smithi138.stdout:1468: writing smithi138138488-29 from 615321 to 1019844 tid 1 2023-06-27T12:44:15.706 INFO:tasks.rados.rados.0.smithi138.stdout:1468: writing smithi138138488-29 from 1804606 to 2261846 tid 2 2023-06-27T12:44:15.714 INFO:tasks.rados.rados.0.smithi138.stdout:1468: writing smithi138138488-29 from 2755658 to 3481883 tid 3 2023-06-27T12:44:15.714 INFO:tasks.rados.rados.0.smithi138.stdout:1469: snap_create 2023-06-27T12:44:15.715 INFO:tasks.rados.rados.0.smithi138.stdout:1463: expect (ObjNum 679 snap 167 seq_num 679) 2023-06-27T12:44:15.910 INFO:tasks.rados.rados.0.smithi138.stdout:1461: finishing write tid 1 to smithi138138488-36 2023-06-27T12:44:15.910 INFO:tasks.rados.rados.0.smithi138.stdout:1462: expect (ObjNum 656 snap 161 seq_num 656) 2023-06-27T12:44:16.065 INFO:tasks.rados.rados.0.smithi138.stdout:1459: finishing write tid 3 to smithi138138488-38 2023-06-27T12:44:16.065 INFO:tasks.rados.rados.0.smithi138.stdout:1459: finishing write tid 4 to smithi138138488-38 2023-06-27T12:44:16.065 INFO:tasks.rados.rados.0.smithi138.stdout:1459: finishing write tid 5 to smithi138138488-38 2023-06-27T12:44:16.065 INFO:tasks.rados.rados.0.smithi138.stdout:1459: oid 38 updating version 0 to 51 2023-06-27T12:44:16.065 INFO:tasks.rados.rados.0.smithi138.stdout:1459: oid 38 updating version 51 to 54 2023-06-27T12:44:16.066 INFO:tasks.rados.rados.0.smithi138.stdout:1459: oid 38 version 54 is already newer than 54 2023-06-27T12:44:16.066 INFO:tasks.rados.rados.0.smithi138.stdout:1459: oid 38 version 54 is already newer than 54 2023-06-27T12:44:16.066 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 54 (ObjNum 698 snap 171 seq_num 698) dirty exists 2023-06-27T12:44:16.066 INFO:tasks.rados.rados.0.smithi138.stdout:1459: left oid 38 (ObjNum 698 snap 171 seq_num 698) 2023-06-27T12:44:16.066 INFO:tasks.rados.rados.0.smithi138.stdout:1465: finishing write tid 1 to smithi138138488-27 2023-06-27T12:44:16.066 INFO:tasks.rados.rados.0.smithi138.stdout:1467: expect (ObjNum 670 snap 165 seq_num 670) 2023-06-27T12:44:16.252 INFO:tasks.rados.rados.0.smithi138.stdout:1461: finishing write tid 2 to smithi138138488-36 2023-06-27T12:44:16.252 INFO:tasks.rados.rados.0.smithi138.stdout:1465: finishing write tid 2 to smithi138138488-27 2023-06-27T12:44:16.252 INFO:tasks.rados.rados.0.smithi138.stdout:1465: finishing write tid 3 to smithi138138488-27 2023-06-27T12:44:16.252 INFO:tasks.rados.rados.0.smithi138.stdout:1465: finishing write tid 4 to smithi138138488-27 2023-06-27T12:44:16.253 INFO:tasks.rados.rados.0.smithi138.stdout:1465: finishing write tid 5 to smithi138138488-27 2023-06-27T12:44:16.253 INFO:tasks.rados.rados.0.smithi138.stdout:1465: oid 27 updating version 0 to 230 2023-06-27T12:44:16.253 INFO:tasks.rados.rados.0.smithi138.stdout:1465: oid 27 version 230 is already newer than 230 2023-06-27T12:44:16.253 INFO:tasks.rados.rados.0.smithi138.stdout:1465: oid 27 version 230 is already newer than 230 2023-06-27T12:44:16.253 INFO:tasks.rados.rados.0.smithi138.stdout:1465: oid 27 version 230 is already newer than 230 2023-06-27T12:44:16.253 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 230 (ObjNum 701 snap 171 seq_num 701) dirty exists 2023-06-27T12:44:16.253 INFO:tasks.rados.rados.0.smithi138.stdout:1465: left oid 27 (ObjNum 701 snap 171 seq_num 701) 2023-06-27T12:44:16.253 INFO:tasks.rados.rados.0.smithi138.stdout:1466: finishing write tid 1 to smithi138138488-22 2023-06-27T12:44:16.254 INFO:tasks.rados.rados.0.smithi138.stdout:1466: finishing write tid 2 to smithi138138488-22 2023-06-27T12:44:16.254 INFO:tasks.rados.rados.0.smithi138.stdout:1461: finishing write tid 3 to smithi138138488-36 2023-06-27T12:44:16.254 INFO:tasks.rados.rados.0.smithi138.stdout:1461: finishing write tid 4 to smithi138138488-36 2023-06-27T12:44:16.254 INFO:tasks.rados.rados.0.smithi138.stdout:1461: finishing write tid 5 to smithi138138488-36 2023-06-27T12:44:16.254 INFO:tasks.rados.rados.0.smithi138.stdout:1461: oid 36 updating version 0 to 285 2023-06-27T12:44:16.254 INFO:tasks.rados.rados.0.smithi138.stdout:1461: oid 36 version 285 is already newer than 285 2023-06-27T12:44:16.254 INFO:tasks.rados.rados.0.smithi138.stdout:1461: oid 36 version 285 is already newer than 285 2023-06-27T12:44:16.254 INFO:tasks.rados.rados.0.smithi138.stdout:1461: oid 36 version 285 is already newer than 285 2023-06-27T12:44:16.255 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 285 (ObjNum 700 snap 171 seq_num 700) dirty exists 2023-06-27T12:44:16.255 INFO:tasks.rados.rados.0.smithi138.stdout:1461: left oid 36 (ObjNum 700 snap 171 seq_num 700) 2023-06-27T12:44:16.255 INFO:tasks.rados.rados.0.smithi138.stdout:1468: finishing write tid 1 to smithi138138488-29 2023-06-27T12:44:16.255 INFO:tasks.rados.rados.0.smithi138.stdout:1468: finishing write tid 2 to smithi138138488-29 2023-06-27T12:44:16.255 INFO:tasks.rados.rados.0.smithi138.stdout:1468: finishing write tid 3 to smithi138138488-29 2023-06-27T12:44:16.255 INFO:tasks.rados.rados.0.smithi138.stdout:1468: finishing write tid 4 to smithi138138488-29 2023-06-27T12:44:16.255 INFO:tasks.rados.rados.0.smithi138.stdout:1468: finishing write tid 5 to smithi138138488-29 2023-06-27T12:44:16.255 INFO:tasks.rados.rados.0.smithi138.stdout:1468: oid 29 updating version 0 to 144 2023-06-27T12:44:16.256 INFO:tasks.rados.rados.0.smithi138.stdout:1468: oid 29 version 144 is already newer than 140 2023-06-27T12:44:16.256 INFO:tasks.rados.rados.0.smithi138.stdout:1468: oid 29 version 144 is already newer than 144 2023-06-27T12:44:16.256 INFO:tasks.rados.rados.0.smithi138.stdout:1468: oid 29 version 144 is already newer than 144 2023-06-27T12:44:16.256 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 144 (ObjNum 703 snap 171 seq_num 703) dirty exists 2023-06-27T12:44:16.256 INFO:tasks.rados.rados.0.smithi138.stdout:1468: left oid 29 (ObjNum 703 snap 171 seq_num 703) 2023-06-27T12:44:16.256 INFO:tasks.rados.rados.0.smithi138.stdout:1466: finishing write tid 3 to smithi138138488-22 2023-06-27T12:44:16.256 INFO:tasks.rados.rados.0.smithi138.stdout:1466: finishing write tid 4 to smithi138138488-22 2023-06-27T12:44:16.257 INFO:tasks.rados.rados.0.smithi138.stdout:1466: finishing write tid 5 to smithi138138488-22 2023-06-27T12:44:16.257 INFO:tasks.rados.rados.0.smithi138.stdout:1466: oid 22 updating version 0 to 236 2023-06-27T12:44:16.257 INFO:tasks.rados.rados.0.smithi138.stdout:1466: oid 22 version 236 is already newer than 236 2023-06-27T12:44:16.257 INFO:tasks.rados.rados.0.smithi138.stdout:1466: oid 22 version 236 is already newer than 236 2023-06-27T12:44:16.257 INFO:tasks.rados.rados.0.smithi138.stdout:1466: oid 22 version 236 is already newer than 236 2023-06-27T12:44:16.257 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 236 (ObjNum 702 snap 171 seq_num 702) dirty exists 2023-06-27T12:44:16.257 INFO:tasks.rados.rados.0.smithi138.stdout:1466: left oid 22 (ObjNum 702 snap 171 seq_num 702) 2023-06-27T12:44:16.541 INFO:tasks.rados.rados.0.smithi138.stdout:1459: done (9 left) 2023-06-27T12:44:16.542 INFO:tasks.rados.rados.0.smithi138.stdout:1461: done (8 left) 2023-06-27T12:44:16.542 INFO:tasks.rados.rados.0.smithi138.stdout:1462: done (7 left) 2023-06-27T12:44:16.542 INFO:tasks.rados.rados.0.smithi138.stdout:1463: done (6 left) 2023-06-27T12:44:16.542 INFO:tasks.rados.rados.0.smithi138.stdout:1464: done (5 left) 2023-06-27T12:44:16.542 INFO:tasks.rados.rados.0.smithi138.stdout:1465: done (4 left) 2023-06-27T12:44:16.542 INFO:tasks.rados.rados.0.smithi138.stdout:1466: done (3 left) 2023-06-27T12:44:16.543 INFO:tasks.rados.rados.0.smithi138.stdout:1467: done (2 left) 2023-06-27T12:44:16.543 INFO:tasks.rados.rados.0.smithi138.stdout:1468: done (1 left) 2023-06-27T12:44:16.543 INFO:tasks.rados.rados.0.smithi138.stdout:1469: done (0 left) 2023-06-27T12:44:16.543 INFO:tasks.rados.rados.0.smithi138.stdout:1470: snap_create 2023-06-27T12:44:16.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:16.738 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:16.733+0000 7f771e31e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:16.741 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:16.737+0000 7f771e31e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:16.786 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:16.788 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:16.789 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:17.541 INFO:tasks.rados.rados.0.smithi138.stdout:1470: done (0 left) 2023-06-27T12:44:17.542 INFO:tasks.rados.rados.0.smithi138.stdout:1471: write (excl) oid 47 current snap is 173 2023-06-27T12:44:17.542 INFO:tasks.rados.rados.0.smithi138.stdout:1471: seq_num 704 ranges {632128=479772,1746735=600898,3066728=653850} 2023-06-27T12:44:17.547 INFO:tasks.rados.rados.0.smithi138.stdout:1471: writing smithi138138488-47 from 632128 to 1111900 tid 1 2023-06-27T12:44:17.554 INFO:tasks.rados.rados.0.smithi138.stdout:1471: writing smithi138138488-47 from 1746735 to 2347633 tid 2 2023-06-27T12:44:17.562 INFO:tasks.rados.rados.0.smithi138.stdout:1471: writing smithi138138488-47 from 3066728 to 3720578 tid 3 2023-06-27T12:44:17.562 INFO:tasks.rados.rados.0.smithi138.stdout:1472: read oid 11 snap 170 2023-06-27T12:44:17.562 INFO:tasks.rados.rados.0.smithi138.stdout:1472: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:44:17.562 INFO:tasks.rados.rados.0.smithi138.stdout:1473: write (excl) oid 42 current snap is 173 2023-06-27T12:44:17.562 INFO:tasks.rados.rados.0.smithi138.stdout:1473: seq_num 705 ranges {648935=555021,1805217=460909,2694151=465122,3817442=31493} 2023-06-27T12:44:17.569 INFO:tasks.rados.rados.0.smithi138.stdout:1473: writing smithi138138488-42 from 648935 to 1203956 tid 1 2023-06-27T12:44:17.575 INFO:tasks.rados.rados.0.smithi138.stdout:1473: writing smithi138138488-42 from 1805217 to 2266126 tid 2 2023-06-27T12:44:17.580 INFO:tasks.rados.rados.0.smithi138.stdout:1473: writing smithi138138488-42 from 2694151 to 3159273 tid 3 2023-06-27T12:44:17.584 INFO:tasks.rados.rados.0.smithi138.stdout:1473: writing smithi138138488-42 from 3817442 to 3848935 tid 4 2023-06-27T12:44:17.584 INFO:tasks.rados.rados.0.smithi138.stdout:1474: snap_create 2023-06-27T12:44:17.622 INFO:tasks.rados.rados.0.smithi138.stdout:1471: finishing write tid 1 to smithi138138488-47 2023-06-27T12:44:17.632 INFO:tasks.rados.rados.0.smithi138.stdout:1473: finishing write tid 1 to smithi138138488-42 2023-06-27T12:44:17.642 INFO:tasks.rados.rados.0.smithi138.stdout:1472: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:44:17.739 INFO:tasks.rados.rados.0.smithi138.stdout:1473: finishing write tid 2 to smithi138138488-42 2023-06-27T12:44:17.739 INFO:tasks.rados.rados.0.smithi138.stdout:1473: finishing write tid 3 to smithi138138488-42 2023-06-27T12:44:17.740 INFO:tasks.rados.rados.0.smithi138.stdout:1473: finishing write tid 4 to smithi138138488-42 2023-06-27T12:44:17.740 INFO:tasks.rados.rados.0.smithi138.stdout:1473: finishing write tid 5 to smithi138138488-42 2023-06-27T12:44:17.740 INFO:tasks.rados.rados.0.smithi138.stdout:1473: finishing write tid 6 to smithi138138488-42 2023-06-27T12:44:17.740 INFO:tasks.rados.rados.0.smithi138.stdout:1473: oid 42 updating version 0 to 193 2023-06-27T12:44:17.741 INFO:tasks.rados.rados.0.smithi138.stdout:1473: oid 42 updating version 193 to 198 2023-06-27T12:44:17.741 INFO:tasks.rados.rados.0.smithi138.stdout:1473: oid 42 version 198 is already newer than 198 2023-06-27T12:44:17.741 INFO:tasks.rados.rados.0.smithi138.stdout:1473: oid 42 version 198 is already newer than 198 2023-06-27T12:44:17.741 INFO:tasks.rados.rados.0.smithi138.stdout:1473: oid 42 version 198 is already newer than 198 2023-06-27T12:44:17.742 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 198 (ObjNum 705 snap 173 seq_num 705) dirty exists 2023-06-27T12:44:17.742 INFO:tasks.rados.rados.0.smithi138.stdout:1473: left oid 42 (ObjNum 705 snap 173 seq_num 705) 2023-06-27T12:44:17.742 INFO:tasks.rados.rados.0.smithi138.stdout:1471: finishing write tid 2 to smithi138138488-47 2023-06-27T12:44:17.742 INFO:tasks.rados.rados.0.smithi138.stdout:1471: finishing write tid 3 to smithi138138488-47 2023-06-27T12:44:17.742 INFO:tasks.rados.rados.0.smithi138.stdout:1471: finishing write tid 4 to smithi138138488-47 2023-06-27T12:44:17.743 INFO:tasks.rados.rados.0.smithi138.stdout:1471: finishing write tid 5 to smithi138138488-47 2023-06-27T12:44:17.743 INFO:tasks.rados.rados.0.smithi138.stdout:1471: oid 47 updating version 0 to 234 2023-06-27T12:44:17.743 INFO:tasks.rados.rados.0.smithi138.stdout:1471: oid 47 version 234 is already newer than 231 2023-06-27T12:44:17.743 INFO:tasks.rados.rados.0.smithi138.stdout:1471: oid 47 version 234 is already newer than 234 2023-06-27T12:44:17.743 INFO:tasks.rados.rados.0.smithi138.stdout:1471: oid 47 version 234 is already newer than 234 2023-06-27T12:44:17.743 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 47 v 234 (ObjNum 704 snap 173 seq_num 704) dirty exists 2023-06-27T12:44:17.743 INFO:tasks.rados.rados.0.smithi138.stdout:1471: left oid 47 (ObjNum 704 snap 173 seq_num 704) 2023-06-27T12:44:17.863 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:17.864 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-06-27T12:44:17.993 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:17.984+0000 7fbeca771700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:17.996 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:17.992+0000 7fbeca771700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:18.549 INFO:tasks.rados.rados.0.smithi138.stdout:1471: done (3 left) 2023-06-27T12:44:18.549 INFO:tasks.rados.rados.0.smithi138.stdout:1472: done (2 left) 2023-06-27T12:44:18.550 INFO:tasks.rados.rados.0.smithi138.stdout:1473: done (1 left) 2023-06-27T12:44:18.550 INFO:tasks.rados.rados.0.smithi138.stdout:1474: done (0 left) 2023-06-27T12:44:18.550 INFO:tasks.rados.rados.0.smithi138.stdout:1475: read oid 14 snap -1 2023-06-27T12:44:18.550 INFO:tasks.rados.rados.0.smithi138.stdout:1475: expect deleted 2023-06-27T12:44:18.550 INFO:tasks.rados.rados.0.smithi138.stdout:1476: read oid 23 snap -1 2023-06-27T12:44:18.550 INFO:tasks.rados.rados.0.smithi138.stdout:1476: expect deleted 2023-06-27T12:44:18.551 INFO:tasks.rados.rados.0.smithi138.stdout:1477: snap_create 2023-06-27T12:44:18.552 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 1 (10000) 2023-06-27T12:44:18.568 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-06-27T12:44:18.698 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:18.689+0000 7f9af24aa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:18.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:18.697+0000 7f9af24aa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:18.790 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:18.917 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:18.912+0000 7fce0c435700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:18.916+0000 7fce0c435700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:18.965 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:18.967 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:18.968 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:19.551 INFO:tasks.rados.rados.0.smithi138.stdout:1475: done (2 left) 2023-06-27T12:44:19.552 INFO:tasks.rados.rados.0.smithi138.stdout:1476: done (1 left) 2023-06-27T12:44:19.552 INFO:tasks.rados.rados.0.smithi138.stdout:1477: done (0 left) 2023-06-27T12:44:19.552 INFO:tasks.rados.rados.0.smithi138.stdout:1478: write oid 17 current snap is 175 2023-06-27T12:44:19.552 INFO:tasks.rados.rados.0.smithi138.stdout:1478: seq_num 706 ranges {665742=630270,1863699=604567,3121574=676394} 2023-06-27T12:44:19.554 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.1 to 1 (10000) 2023-06-27T12:44:19.558 INFO:tasks.rados.rados.0.smithi138.stdout:1478: writing smithi138138488-17 from 665742 to 1296012 tid 1 2023-06-27T12:44:19.565 INFO:tasks.rados.rados.0.smithi138.stdout:1478: writing smithi138138488-17 from 1863699 to 2468266 tid 2 2023-06-27T12:44:19.567 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-06-27T12:44:19.572 INFO:tasks.rados.rados.0.smithi138.stdout:1478: writing smithi138138488-17 from 3121574 to 3797968 tid 3 2023-06-27T12:44:19.573 INFO:tasks.rados.rados.0.smithi138.stdout:1479: snap_create 2023-06-27T12:44:19.648 INFO:tasks.rados.rados.0.smithi138.stdout:1478: finishing write tid 1 to smithi138138488-17 2023-06-27T12:44:19.649 INFO:tasks.rados.rados.0.smithi138.stdout:1478: finishing write tid 2 to smithi138138488-17 2023-06-27T12:44:19.650 INFO:tasks.rados.rados.0.smithi138.stdout:1478: finishing write tid 3 to smithi138138488-17 2023-06-27T12:44:19.650 INFO:tasks.rados.rados.0.smithi138.stdout:1478: finishing write tid 4 to smithi138138488-17 2023-06-27T12:44:19.652 INFO:tasks.rados.rados.0.smithi138.stdout:1478: finishing write tid 5 to smithi138138488-17 2023-06-27T12:44:19.652 INFO:tasks.rados.rados.0.smithi138.stdout:1478: oid 17 updating version 0 to 239 2023-06-27T12:44:19.653 INFO:tasks.rados.rados.0.smithi138.stdout:1478: oid 17 version 239 is already newer than 239 2023-06-27T12:44:19.653 INFO:tasks.rados.rados.0.smithi138.stdout:1478: oid 17 version 239 is already newer than 239 2023-06-27T12:44:19.653 INFO:tasks.rados.rados.0.smithi138.stdout:1478: oid 17 version 239 is already newer than 239 2023-06-27T12:44:19.654 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 239 (ObjNum 706 snap 175 seq_num 706) dirty exists 2023-06-27T12:44:19.654 INFO:tasks.rados.rados.0.smithi138.stdout:1478: left oid 17 (ObjNum 706 snap 175 seq_num 706) 2023-06-27T12:44:19.704 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:19.694+0000 7fb43e85d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:19.706 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:19.703+0000 7fb43e85d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:20.462 INFO:tasks.rados.rados.0.smithi138.stdout:1478: done (1 left) 2023-06-27T12:44:20.462 INFO:tasks.rados.rados.0.smithi138.stdout:1479: done (0 left) 2023-06-27T12:44:20.462 INFO:tasks.rados.rados.0.smithi138.stdout:1480: write (excl) oid 3 current snap is 176 2023-06-27T12:44:20.462 INFO:tasks.rados.rados.0.smithi138.stdout:1480: seq_num 707 ranges {682549=705519,1922181=464578,3148997=487666} 2023-06-27T12:44:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.2 to 1 (10000) 2023-06-27T12:44:20.473 INFO:tasks.rados.rados.0.smithi138.stdout:1480: writing smithi138138488-3 from 682549 to 1388068 tid 1 2023-06-27T12:44:20.478 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-06-27T12:44:20.479 INFO:tasks.rados.rados.0.smithi138.stdout:1480: writing smithi138138488-3 from 1922181 to 2386759 tid 2 2023-06-27T12:44:20.486 INFO:tasks.rados.rados.0.smithi138.stdout:1480: writing smithi138138488-3 from 3148997 to 3636663 tid 3 2023-06-27T12:44:20.486 INFO:tasks.rados.rados.0.smithi138.stdout:1481: write (excl) oid 12 current snap is 176 2023-06-27T12:44:20.487 INFO:tasks.rados.rados.0.smithi138.stdout:1481: seq_num 708 ranges {699356=497121,1980663=608236,3176420=415291} 2023-06-27T12:44:20.493 INFO:tasks.rados.rados.0.smithi138.stdout:1481: writing smithi138138488-12 from 699356 to 1196477 tid 1 2023-06-27T12:44:20.501 INFO:tasks.rados.rados.0.smithi138.stdout:1481: writing smithi138138488-12 from 1980663 to 2588899 tid 2 2023-06-27T12:44:20.507 INFO:tasks.rados.rados.0.smithi138.stdout:1481: writing smithi138138488-12 from 3176420 to 3591711 tid 3 2023-06-27T12:44:20.508 INFO:tasks.rados.rados.0.smithi138.stdout:1482: write oid 21 current snap is 176 2023-06-27T12:44:20.508 INFO:tasks.rados.rados.0.smithi138.stdout:1480: finishing write tid 1 to smithi138138488-3 2023-06-27T12:44:20.508 INFO:tasks.rados.rados.0.smithi138.stdout:1482: seq_num 709 ranges {716163=572370,2039145=468247,3203843=626563} 2023-06-27T12:44:20.516 INFO:tasks.rados.rados.0.smithi138.stdout:1482: writing smithi138138488-21 from 716163 to 1288533 tid 1 2023-06-27T12:44:20.523 INFO:tasks.rados.rados.0.smithi138.stdout:1482: writing smithi138138488-21 from 2039145 to 2507392 tid 2 2023-06-27T12:44:20.531 INFO:tasks.rados.rados.0.smithi138.stdout:1482: writing smithi138138488-21 from 3203843 to 3830406 tid 3 2023-06-27T12:44:20.531 INFO:tasks.rados.rados.0.smithi138.stdout:1483: write oid 16 current snap is 176 2023-06-27T12:44:20.531 INFO:tasks.rados.rados.0.smithi138.stdout:1483: seq_num 710 ranges {732970=647619,2097627=611905,3231266=437835} 2023-06-27T12:44:20.539 INFO:tasks.rados.rados.0.smithi138.stdout:1483: writing smithi138138488-16 from 732970 to 1380589 tid 1 2023-06-27T12:44:20.547 INFO:tasks.rados.rados.0.smithi138.stdout:1483: writing smithi138138488-16 from 2097627 to 2709532 tid 2 2023-06-27T12:44:20.553 INFO:tasks.rados.rados.0.smithi138.stdout:1483: writing smithi138138488-16 from 3231266 to 3669101 tid 3 2023-06-27T12:44:20.553 INFO:tasks.rados.rados.0.smithi138.stdout:1484: delete oid 18 current snap is 176 2023-06-27T12:44:20.553 INFO:tasks.rados.rados.0.smithi138.stdout:1480: finishing write tid 2 to smithi138138488-3 2023-06-27T12:44:20.553 INFO:tasks.rados.rados.0.smithi138.stdout:1480: finishing write tid 3 to smithi138138488-3 2023-06-27T12:44:20.553 INFO:tasks.rados.rados.0.smithi138.stdout:1480: finishing write tid 4 to smithi138138488-3 2023-06-27T12:44:20.554 INFO:tasks.rados.rados.0.smithi138.stdout:1480: finishing write tid 5 to smithi138138488-3 2023-06-27T12:44:20.554 INFO:tasks.rados.rados.0.smithi138.stdout:1480: oid 3 updating version 0 to 219 2023-06-27T12:44:20.554 INFO:tasks.rados.rados.0.smithi138.stdout:1480: oid 3 version 219 is already newer than 219 2023-06-27T12:44:20.554 INFO:tasks.rados.rados.0.smithi138.stdout:1480: oid 3 version 219 is already newer than 219 2023-06-27T12:44:20.554 INFO:tasks.rados.rados.0.smithi138.stdout:1480: oid 3 version 219 is already newer than 215 2023-06-27T12:44:20.554 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 219 (ObjNum 707 snap 176 seq_num 707) dirty exists 2023-06-27T12:44:20.554 INFO:tasks.rados.rados.0.smithi138.stdout:1480: left oid 3 (ObjNum 707 snap 176 seq_num 707) 2023-06-27T12:44:20.632 INFO:tasks.rados.rados.0.smithi138.stdout:1481: finishing write tid 1 to smithi138138488-12 2023-06-27T12:44:20.649 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:20.639+0000 7f473aadd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:20.653 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:20.650+0000 7f473aadd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:20.681 INFO:tasks.rados.rados.0.smithi138.stdout:1482: finishing write tid 1 to smithi138138488-21 2023-06-27T12:44:20.686 INFO:tasks.rados.rados.0.smithi138.stdout:1482: finishing write tid 2 to smithi138138488-21 2023-06-27T12:44:20.692 INFO:tasks.rados.rados.0.smithi138.stdout:1483: finishing write tid 1 to smithi138138488-16 2023-06-27T12:44:20.693 INFO:tasks.rados.rados.0.smithi138.stdout:1483: finishing write tid 2 to smithi138138488-16 2023-06-27T12:44:20.701 INFO:tasks.rados.rados.0.smithi138.stdout:1482: finishing write tid 3 to smithi138138488-21 2023-06-27T12:44:20.703 INFO:tasks.rados.rados.0.smithi138.stdout:1482: finishing write tid 4 to smithi138138488-21 2023-06-27T12:44:20.706 INFO:tasks.rados.rados.0.smithi138.stdout:1483: finishing write tid 3 to smithi138138488-16 2023-06-27T12:44:20.707 INFO:tasks.rados.rados.0.smithi138.stdout:1483: finishing write tid 4 to smithi138138488-16 2023-06-27T12:44:20.708 INFO:tasks.rados.rados.0.smithi138.stdout:1483: finishing write tid 5 to smithi138138488-16 2023-06-27T12:44:20.708 INFO:tasks.rados.rados.0.smithi138.stdout:1483: oid 16 updating version 0 to 427 2023-06-27T12:44:20.708 INFO:tasks.rados.rados.0.smithi138.stdout:1483: oid 16 version 427 is already newer than 427 2023-06-27T12:44:20.708 INFO:tasks.rados.rados.0.smithi138.stdout:1483: oid 16 version 427 is already newer than 427 2023-06-27T12:44:20.708 INFO:tasks.rados.rados.0.smithi138.stdout:1483: oid 16 version 427 is already newer than 427 2023-06-27T12:44:20.708 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 427 (ObjNum 710 snap 176 seq_num 710) dirty exists 2023-06-27T12:44:20.709 INFO:tasks.rados.rados.0.smithi138.stdout:1483: left oid 16 (ObjNum 710 snap 176 seq_num 710) 2023-06-27T12:44:20.709 INFO:tasks.rados.rados.0.smithi138.stdout:1482: finishing write tid 5 to smithi138138488-21 2023-06-27T12:44:20.710 INFO:tasks.rados.rados.0.smithi138.stdout:1482: oid 21 updating version 0 to 409 2023-06-27T12:44:20.710 INFO:tasks.rados.rados.0.smithi138.stdout:1482: oid 21 version 409 is already newer than 409 2023-06-27T12:44:20.710 INFO:tasks.rados.rados.0.smithi138.stdout:1482: oid 21 version 409 is already newer than 409 2023-06-27T12:44:20.710 INFO:tasks.rados.rados.0.smithi138.stdout:1482: oid 21 version 409 is already newer than 409 2023-06-27T12:44:20.710 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 21 v 409 (ObjNum 709 snap 176 seq_num 709) dirty exists 2023-06-27T12:44:20.710 INFO:tasks.rados.rados.0.smithi138.stdout:1482: left oid 21 (ObjNum 709 snap 176 seq_num 709) 2023-06-27T12:44:20.741 INFO:tasks.rados.rados.0.smithi138.stdout:1481: finishing write tid 2 to smithi138138488-12 2023-06-27T12:44:20.744 INFO:tasks.rados.rados.0.smithi138.stdout:1481: finishing write tid 3 to smithi138138488-12 2023-06-27T12:44:20.745 INFO:tasks.rados.rados.0.smithi138.stdout:1481: finishing write tid 4 to smithi138138488-12 2023-06-27T12:44:20.747 INFO:tasks.rados.rados.0.smithi138.stdout:1481: finishing write tid 5 to smithi138138488-12 2023-06-27T12:44:20.748 INFO:tasks.rados.rados.0.smithi138.stdout:1481: oid 12 updating version 0 to 289 2023-06-27T12:44:20.748 INFO:tasks.rados.rados.0.smithi138.stdout:1481: oid 12 version 289 is already newer than 289 2023-06-27T12:44:20.748 INFO:tasks.rados.rados.0.smithi138.stdout:1481: oid 12 version 289 is already newer than 289 2023-06-27T12:44:20.748 INFO:tasks.rados.rados.0.smithi138.stdout:1481: oid 12 version 289 is already newer than 286 2023-06-27T12:44:20.748 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 289 (ObjNum 708 snap 176 seq_num 708) dirty exists 2023-06-27T12:44:20.749 INFO:tasks.rados.rados.0.smithi138.stdout:1481: left oid 12 (ObjNum 708 snap 176 seq_num 708) 2023-06-27T12:44:20.749 INFO:tasks.rados.rados.0.smithi138.stdout:1480: done (4 left) 2023-06-27T12:44:20.749 INFO:tasks.rados.rados.0.smithi138.stdout:1481: done (3 left) 2023-06-27T12:44:20.749 INFO:tasks.rados.rados.0.smithi138.stdout:1482: done (2 left) 2023-06-27T12:44:20.749 INFO:tasks.rados.rados.0.smithi138.stdout:1483: done (1 left) 2023-06-27T12:44:20.750 INFO:tasks.rados.rados.0.smithi138.stdout:1484: done (0 left) 2023-06-27T12:44:20.750 INFO:tasks.rados.rados.0.smithi138.stdout:1485: snap_create 2023-06-27T12:44:20.970 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:20.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~226s 2023-06-27T12:44:21.096 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:21.091+0000 7f466ecc7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:21.099 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:21.095+0000 7f466ecc7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:21.145 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:21.148 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:21.149 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:21.565 INFO:tasks.rados.rados.0.smithi138.stdout:1485: done (0 left) 2023-06-27T12:44:21.565 INFO:tasks.rados.rados.0.smithi138.stdout:1486: write (excl) oid 45 current snap is 177 2023-06-27T12:44:21.565 INFO:tasks.rados.rados.0.smithi138.stdout:1486: seq_num 712 ranges {766584=798117,2098238=615574,3169759=460379} 2023-06-27T12:44:21.567 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.3 to 1 (10000) 2023-06-27T12:44:21.573 INFO:tasks.rados.rados.0.smithi138.stdout:1486: writing smithi138138488-45 from 766584 to 1564701 tid 1 2023-06-27T12:44:21.580 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-06-27T12:44:21.581 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:21.581 INFO:tasks.rados.rados.0.smithi138.stdout:1486: writing smithi138138488-45 from 2098238 to 2713812 tid 2 2023-06-27T12:44:21.585 INFO:tasks.rados.rados.0.smithi138.stdout:1486: writing smithi138138488-45 from 3169759 to 3630138 tid 3 2023-06-27T12:44:21.586 INFO:tasks.rados.rados.0.smithi138.stdout:1487: snap_remove snap 169 2023-06-27T12:44:21.712 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:21.705+0000 7f64bf242700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:21.715 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:21.711+0000 7f64bf242700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:21.924 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:21.926 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:21.938 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2023-06-27T12:44:21.560090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103367274,"num_objects":371,"num_object_clones":324,"num_object_copies":1113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":2152,"num_read_kb":1581179,"num_write":1955,"num_write_kb":572681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3567,"ondisk_log_size":3567,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1083644,"kb_used_data":947676,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376403716,"statfs":{"total":386547056640,"available":385437405184,"internally_reserved":0,"allocated":970420224,"data_stored":1117153998,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10571900,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":41,"num_read_kb":33365,"num_write":36,"num_write_kb":12122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023601"},"pg_stats":[{"pgid":"3.f","version":"361'54","reported_seq":868,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.563042+0000","last_change":"2023-06-27T12:44:15.568908+0000","last_active":"2023-06-27T12:44:18.563042+0000","last_peered":"2023-06-27T12:44:18.563042+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:18.563042+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3731286,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":13,"num_read_kb":7,"num_write":33,"num_write_kb":9288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"363'234","reported_seq":1150,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.564643+0000","last_change":"2023-06-27T12:44:14.572505+0000","last_active":"2023-06-27T12:44:18.564643+0000","last_peered":"2023-06-27T12:44:18.564643+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:18.564643+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336899,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":204991,"num_write":167,"num_write_kb":50732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"361'285","reported_seq":1123,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.572508+0000","last_change":"2023-06-27T12:44:14.562818+0000","last_active":"2023-06-27T12:44:18.572508+0000","last_peered":"2023-06-27T12:44:18.572508+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:18.572508+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5966716,"num_objects":33,"num_object_clones":28,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":252,"num_read_kb":171532,"num_write":179,"num_write_kb":52451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":33}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"363'198","reported_seq":1073,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.574212+0000","last_change":"2023-06-27T12:44:15.607230+0000","last_active":"2023-06-27T12:44:18.574212+0000","last_peered":"2023-06-27T12:44:18.574212+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:18.574212+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10353508,"num_objects":29,"num_object_clones":25,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":175,"num_read_kb":119557,"num_write":135,"num_write_kb":38638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":29}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":612,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.575732+0000","last_change":"2023-06-27T12:44:15.566352+0000","last_active":"2023-06-27T12:44:18.575732+0000","last_peered":"2023-06-27T12:44:18.575732+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:18.575732+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"358'420","reported_seq":1147,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.574820+0000","last_change":"2023-06-27T12:44:15.649635+0000","last_active":"2023-06-27T12:44:18.574820+0000","last_peered":"2023-06-27T12:44:18.574820+0000","last_clean":"2023-06-27T12:44:18.574820+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:18.574820+0000","last_undegraded":"2023-06-27T12:44:18.574820+0000","last_fullsized":"2023-06-27T12:44:18.574820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6118012,"num_objects":46,"num_object_clones":40,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":313,"num_read_kb":193215,"num_write":243,"num_write_kb":69875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"352'203","reported_seq":960,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.567453+0000","last_change":"2023-06-27T12:44:15.606567+0000","last_active":"2023-06-27T12:44:18.567453+0000","last_peered":"2023-06-27T12:44:18.567453+0000","last_clean":"2023-06-27T12:44:18.567453+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:18.567453+0000","last_undegraded":"2023-06-27T12:44:18.567453+0000","last_fullsized":"2023-06-27T12:44:18.567453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":132,"num_read_kb":85550,"num_write":127,"num_write_kb":37347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":900,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.569695+0000","last_change":"2023-06-27T12:44:14.560605+0000","last_active":"2023-06-27T12:44:18.569695+0000","last_peered":"2023-06-27T12:44:18.569695+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:18.569695+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":612,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.570929+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:18.570929+0000","last_peered":"2023-06-27T12:44:18.570929+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:18.570929+0000","last_undegraded":"2023-06-27T12:44:18.570929+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":808,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.570545+0000","last_change":"2023-06-27T12:44:14.564748+0000","last_active":"2023-06-27T12:44:18.570545+0000","last_peered":"2023-06-27T12:44:18.570545+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:18.570545+0000","last_undegraded":"2023-06-27T12:44:18.570545+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":593,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.573776+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:18.573776+0000","last_peered":"2023-06-27T12:44:18.573776+0000","last_clean":"2023-06-27T12:44:18.573776+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:18.573776+0000","last_undegraded":"2023-06-27T12:44:18.573776+0000","last_fullsized":"2023-06-27T12:44:18.573776+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"349'194","reported_seq":1038,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.563396+0000","last_change":"2023-06-27T12:44:14.552049+0000","last_active":"2023-06-27T12:44:18.563396+0000","last_peered":"2023-06-27T12:44:18.563396+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:18.563396+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6101205,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":80,"num_read_kb":70438,"num_write":124,"num_write_kb":34515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":576,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.559266+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:18.559266+0000","last_peered":"2023-06-27T12:44:18.559266+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:18.559266+0000","last_undegraded":"2023-06-27T12:44:18.559266+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"354'447","reported_seq":719,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.562227+0000","last_change":"2023-06-27T12:44:14.551270+0000","last_active":"2023-06-27T12:44:18.562227+0000","last_peered":"2023-06-27T12:44:18.562227+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:18.562227+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5865907,"num_objects":26,"num_object_clones":24,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":41842,"num_write":131,"num_write_kb":40677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":573,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.571144+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:18.571144+0000","last_peered":"2023-06-27T12:44:18.571144+0000","last_clean":"2023-06-27T12:44:18.571144+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:18.571144+0000","last_undegraded":"2023-06-27T12:44:18.571144+0000","last_fullsized":"2023-06-27T12:44:18.571144+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":345,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.560082+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:18.560082+0000","last_peered":"2023-06-27T12:44:18.560082+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:18.560082+0000","last_undegraded":"2023-06-27T12:44:18.560082+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"354'401","reported_seq":1254,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.567051+0000","last_change":"2023-06-27T12:44:14.557999+0000","last_active":"2023-06-27T12:44:18.567051+0000","last_peered":"2023-06-27T12:44:18.567051+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:18.567051+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790292,"num_objects":51,"num_object_clones":45,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":319,"num_read_kb":198252,"num_write":283,"num_write_kb":83218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":593,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.566866+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:18.566866+0000","last_peered":"2023-06-27T12:44:18.566866+0000","last_clean":"2023-06-27T12:44:18.566866+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:18.566866+0000","last_undegraded":"2023-06-27T12:44:18.566866+0000","last_fullsized":"2023-06-27T12:44:18.566866+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"354'257","reported_seq":646,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.575821+0000","last_change":"2023-06-27T12:44:14.563669+0000","last_active":"2023-06-27T12:44:18.575821+0000","last_peered":"2023-06-27T12:44:18.575821+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:18.575821+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9866138,"num_objects":19,"num_object_clones":19,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59443,"num_write":105,"num_write_kb":30476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":355,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.562483+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:18.562483+0000","last_peered":"2023-06-27T12:44:18.562483+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:18.562483+0000","last_undegraded":"2023-06-27T12:44:18.562483+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"353'214","reported_seq":1083,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.569381+0000","last_change":"2023-06-27T12:44:14.548447+0000","last_active":"2023-06-27T12:44:18.569381+0000","last_peered":"2023-06-27T12:44:18.569381+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:18.569381+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8588773,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":114,"num_read_kb":123169,"num_write":141,"num_write_kb":40461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"361'144","reported_seq":602,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.569732+0000","last_change":"2023-06-27T12:44:14.558520+0000","last_active":"2023-06-27T12:44:18.569732+0000","last_peered":"2023-06-27T12:44:18.569732+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:18.569732+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3815321,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":101,"num_read_kb":56967,"num_write":58,"num_write_kb":16387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":612,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.564741+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:18.564741+0000","last_peered":"2023-06-27T12:44:18.564741+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:18.564741+0000","last_undegraded":"2023-06-27T12:44:18.564741+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":573,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.572870+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:18.572870+0000","last_peered":"2023-06-27T12:44:18.572870+0000","last_clean":"2023-06-27T12:44:18.572870+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:18.572870+0000","last_undegraded":"2023-06-27T12:44:18.572870+0000","last_fullsized":"2023-06-27T12:44:18.572870+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"361'236","reported_seq":1081,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.561173+0000","last_change":"2023-06-27T12:44:14.548401+0000","last_active":"2023-06-27T12:44:18.561173+0000","last_peered":"2023-06-27T12:44:18.561173+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:18.561173+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5916328,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":122,"num_read_kb":132246,"num_write":125,"num_write_kb":37073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":103367274,"num_objects":371,"num_object_clones":324,"num_object_copies":1113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":2152,"num_read_kb":1581179,"num_write":1955,"num_write_kb":572681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3470,"ondisk_log_size":3470,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673345,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":287048,"kb_used_data":252032,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94084792,"statfs":{"total":96636764160,"available":96342827008,"internally_reserved":0,"allocated":258080768,"data_stored":298525801,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":293764,"kb_used_data":258748,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94078076,"statfs":{"total":96636764160,"available":96335949824,"internally_reserved":0,"allocated":264957952,"data_stored":302705379,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673345,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":321552,"kb_used_data":286536,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94050288,"statfs":{"total":96636764160,"available":96307494912,"internally_reserved":0,"allocated":293412864,"data_stored":338883266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:21.940 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:22.107 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:22.100+0000 7ff7ec4ec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:22.111 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:22.107+0000 7ff7ec4ec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:22.303 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:22.305 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:22.316 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2023-06-27T12:44:21.995360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103367274,"num_objects":371,"num_object_clones":324,"num_object_copies":1113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":2152,"num_read_kb":1581179,"num_write":1955,"num_write_kb":572681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3567,"ondisk_log_size":3567,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1083644,"kb_used_data":947676,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376403716,"statfs":{"total":386547056640,"available":385437405184,"internally_reserved":0,"allocated":970420224,"data_stored":1117153998,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10571900,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":41,"num_read_kb":33365,"num_write":36,"num_write_kb":12122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001312"},"pg_stats":[{"pgid":"3.f","version":"361'54","reported_seq":868,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.563042+0000","last_change":"2023-06-27T12:44:15.568908+0000","last_active":"2023-06-27T12:44:18.563042+0000","last_peered":"2023-06-27T12:44:18.563042+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:18.563042+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3731286,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":13,"num_read_kb":7,"num_write":33,"num_write_kb":9288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"363'234","reported_seq":1150,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.564643+0000","last_change":"2023-06-27T12:44:14.572505+0000","last_active":"2023-06-27T12:44:18.564643+0000","last_peered":"2023-06-27T12:44:18.564643+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:18.564643+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336899,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":204991,"num_write":167,"num_write_kb":50732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"361'285","reported_seq":1123,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.572508+0000","last_change":"2023-06-27T12:44:14.562818+0000","last_active":"2023-06-27T12:44:18.572508+0000","last_peered":"2023-06-27T12:44:18.572508+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:18.572508+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5966716,"num_objects":33,"num_object_clones":28,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":252,"num_read_kb":171532,"num_write":179,"num_write_kb":52451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":33}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"363'198","reported_seq":1073,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.574212+0000","last_change":"2023-06-27T12:44:15.607230+0000","last_active":"2023-06-27T12:44:18.574212+0000","last_peered":"2023-06-27T12:44:18.574212+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:18.574212+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10353508,"num_objects":29,"num_object_clones":25,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":175,"num_read_kb":119557,"num_write":135,"num_write_kb":38638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":29}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":612,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.575732+0000","last_change":"2023-06-27T12:44:15.566352+0000","last_active":"2023-06-27T12:44:18.575732+0000","last_peered":"2023-06-27T12:44:18.575732+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:18.575732+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"358'420","reported_seq":1147,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.574820+0000","last_change":"2023-06-27T12:44:15.649635+0000","last_active":"2023-06-27T12:44:18.574820+0000","last_peered":"2023-06-27T12:44:18.574820+0000","last_clean":"2023-06-27T12:44:18.574820+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:18.574820+0000","last_undegraded":"2023-06-27T12:44:18.574820+0000","last_fullsized":"2023-06-27T12:44:18.574820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6118012,"num_objects":46,"num_object_clones":40,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":313,"num_read_kb":193215,"num_write":243,"num_write_kb":69875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"352'203","reported_seq":960,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.567453+0000","last_change":"2023-06-27T12:44:15.606567+0000","last_active":"2023-06-27T12:44:18.567453+0000","last_peered":"2023-06-27T12:44:18.567453+0000","last_clean":"2023-06-27T12:44:18.567453+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:18.567453+0000","last_undegraded":"2023-06-27T12:44:18.567453+0000","last_fullsized":"2023-06-27T12:44:18.567453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":132,"num_read_kb":85550,"num_write":127,"num_write_kb":37347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":900,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.569695+0000","last_change":"2023-06-27T12:44:14.560605+0000","last_active":"2023-06-27T12:44:18.569695+0000","last_peered":"2023-06-27T12:44:18.569695+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:18.569695+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":612,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.570929+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:18.570929+0000","last_peered":"2023-06-27T12:44:18.570929+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:18.570929+0000","last_undegraded":"2023-06-27T12:44:18.570929+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":808,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.570545+0000","last_change":"2023-06-27T12:44:14.564748+0000","last_active":"2023-06-27T12:44:18.570545+0000","last_peered":"2023-06-27T12:44:18.570545+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:18.570545+0000","last_undegraded":"2023-06-27T12:44:18.570545+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":593,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.573776+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:18.573776+0000","last_peered":"2023-06-27T12:44:18.573776+0000","last_clean":"2023-06-27T12:44:18.573776+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:18.573776+0000","last_undegraded":"2023-06-27T12:44:18.573776+0000","last_fullsized":"2023-06-27T12:44:18.573776+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"349'194","reported_seq":1038,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.563396+0000","last_change":"2023-06-27T12:44:14.552049+0000","last_active":"2023-06-27T12:44:18.563396+0000","last_peered":"2023-06-27T12:44:18.563396+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:18.563396+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6101205,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":80,"num_read_kb":70438,"num_write":124,"num_write_kb":34515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":576,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.559266+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:18.559266+0000","last_peered":"2023-06-27T12:44:18.559266+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:18.559266+0000","last_undegraded":"2023-06-27T12:44:18.559266+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"354'447","reported_seq":719,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.562227+0000","last_change":"2023-06-27T12:44:14.551270+0000","last_active":"2023-06-27T12:44:18.562227+0000","last_peered":"2023-06-27T12:44:18.562227+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:18.562227+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5865907,"num_objects":26,"num_object_clones":24,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":41842,"num_write":131,"num_write_kb":40677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":573,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.571144+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:18.571144+0000","last_peered":"2023-06-27T12:44:18.571144+0000","last_clean":"2023-06-27T12:44:18.571144+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:18.571144+0000","last_undegraded":"2023-06-27T12:44:18.571144+0000","last_fullsized":"2023-06-27T12:44:18.571144+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":345,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.560082+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:18.560082+0000","last_peered":"2023-06-27T12:44:18.560082+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:18.560082+0000","last_undegraded":"2023-06-27T12:44:18.560082+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"354'401","reported_seq":1254,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.567051+0000","last_change":"2023-06-27T12:44:14.557999+0000","last_active":"2023-06-27T12:44:18.567051+0000","last_peered":"2023-06-27T12:44:18.567051+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:18.567051+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790292,"num_objects":51,"num_object_clones":45,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":319,"num_read_kb":198252,"num_write":283,"num_write_kb":83218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":593,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.566866+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:18.566866+0000","last_peered":"2023-06-27T12:44:18.566866+0000","last_clean":"2023-06-27T12:44:18.566866+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:18.566866+0000","last_undegraded":"2023-06-27T12:44:18.566866+0000","last_fullsized":"2023-06-27T12:44:18.566866+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"354'257","reported_seq":646,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.575821+0000","last_change":"2023-06-27T12:44:14.563669+0000","last_active":"2023-06-27T12:44:18.575821+0000","last_peered":"2023-06-27T12:44:18.575821+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:18.575821+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9866138,"num_objects":19,"num_object_clones":19,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59443,"num_write":105,"num_write_kb":30476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":355,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.562483+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:18.562483+0000","last_peered":"2023-06-27T12:44:18.562483+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:18.562483+0000","last_undegraded":"2023-06-27T12:44:18.562483+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"353'214","reported_seq":1083,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.569381+0000","last_change":"2023-06-27T12:44:14.548447+0000","last_active":"2023-06-27T12:44:18.569381+0000","last_peered":"2023-06-27T12:44:18.569381+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:18.569381+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8588773,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":114,"num_read_kb":123169,"num_write":141,"num_write_kb":40461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"361'144","reported_seq":602,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.569732+0000","last_change":"2023-06-27T12:44:14.558520+0000","last_active":"2023-06-27T12:44:18.569732+0000","last_peered":"2023-06-27T12:44:18.569732+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:18.569732+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3815321,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":101,"num_read_kb":56967,"num_write":58,"num_write_kb":16387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":612,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.564741+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:18.564741+0000","last_peered":"2023-06-27T12:44:18.564741+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:18.564741+0000","last_undegraded":"2023-06-27T12:44:18.564741+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":573,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.572870+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:18.572870+0000","last_peered":"2023-06-27T12:44:18.572870+0000","last_clean":"2023-06-27T12:44:18.572870+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:18.572870+0000","last_undegraded":"2023-06-27T12:44:18.572870+0000","last_fullsized":"2023-06-27T12:44:18.572870+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"361'236","reported_seq":1081,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.561173+0000","last_change":"2023-06-27T12:44:14.548401+0000","last_active":"2023-06-27T12:44:18.561173+0000","last_peered":"2023-06-27T12:44:18.561173+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:18.561173+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5916328,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":122,"num_read_kb":132246,"num_write":125,"num_write_kb":37073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":103367274,"num_objects":371,"num_object_clones":324,"num_object_copies":1113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":2152,"num_read_kb":1581179,"num_write":1955,"num_write_kb":572681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3470,"ondisk_log_size":3470,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673345,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":287048,"kb_used_data":252032,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94084792,"statfs":{"total":96636764160,"available":96342827008,"internally_reserved":0,"allocated":258080768,"data_stored":298525801,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":293764,"kb_used_data":258748,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94078076,"statfs":{"total":96636764160,"available":96335949824,"internally_reserved":0,"allocated":264957952,"data_stored":302705379,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673345,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":321552,"kb_used_data":286536,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94050288,"statfs":{"total":96636764160,"available":96307494912,"internally_reserved":0,"allocated":293412864,"data_stored":338883266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:22.318 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-06-27T12:44:22.319 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:22.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:22.445 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:22.438+0000 7fb93cb74700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:22.448 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:22.444+0000 7fb93cb74700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:22.569 INFO:tasks.rados.rados.0.smithi138.stdout:1487: done (1 left) 2023-06-27T12:44:22.569 INFO:tasks.rados.rados.0.smithi138.stdout:1488: write oid 31 current snap is 177 2023-06-27T12:44:22.569 INFO:tasks.rados.rados.0.smithi138.stdout:1486: finishing write tid 1 to smithi138138488-45 2023-06-27T12:44:22.569 INFO:tasks.rados.rados.0.smithi138.stdout:1488: seq_num 713 ranges {783391=473366,1756720=475585,2797182=788004} 2023-06-27T12:44:22.575 INFO:tasks.rados.rados.0.smithi138.stdout:1488: writing smithi138138488-31 from 783391 to 1256757 tid 1 2023-06-27T12:44:22.581 INFO:tasks.rados.rados.0.smithi138.stdout:1488: writing smithi138138488-31 from 1756720 to 2232305 tid 2 2023-06-27T12:44:22.589 INFO:tasks.rados.rados.0.smithi138.stdout:1488: writing smithi138138488-31 from 2797182 to 3585186 tid 3 2023-06-27T12:44:22.590 INFO:tasks.rados.rados.0.smithi138.stdout:1489: read oid 40 snap -1 2023-06-27T12:44:22.590 INFO:tasks.rados.rados.0.smithi138.stdout:1489: expect (ObjNum 679 snap 167 seq_num 679) 2023-06-27T12:44:22.590 INFO:tasks.rados.rados.0.smithi138.stdout:1490: read oid 24 snap -1 2023-06-27T12:44:22.590 INFO:tasks.rados.rados.0.smithi138.stdout:1490: expect deleted 2023-06-27T12:44:22.590 INFO:tasks.rados.rados.0.smithi138.stdout:1491: delete oid 9 current snap is 177 2023-06-27T12:44:22.590 INFO:tasks.rados.rados.0.smithi138.stdout:1486: finishing write tid 2 to smithi138138488-45 2023-06-27T12:44:22.590 INFO:tasks.rados.rados.0.smithi138.stdout:1486: finishing write tid 3 to smithi138138488-45 2023-06-27T12:44:22.591 INFO:tasks.rados.rados.0.smithi138.stdout:1486: finishing write tid 4 to smithi138138488-45 2023-06-27T12:44:22.591 INFO:tasks.rados.rados.0.smithi138.stdout:1486: finishing write tid 5 to smithi138138488-45 2023-06-27T12:44:22.591 INFO:tasks.rados.rados.0.smithi138.stdout:1486: oid 45 updating version 0 to 451 2023-06-27T12:44:22.591 INFO:tasks.rados.rados.0.smithi138.stdout:1486: oid 45 version 451 is already newer than 451 2023-06-27T12:44:22.591 INFO:tasks.rados.rados.0.smithi138.stdout:1486: oid 45 version 451 is already newer than 448 2023-06-27T12:44:22.591 INFO:tasks.rados.rados.0.smithi138.stdout:1486: oid 45 version 451 is already newer than 451 2023-06-27T12:44:22.591 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 451 (ObjNum 712 snap 177 seq_num 712) dirty exists 2023-06-27T12:44:22.591 INFO:tasks.rados.rados.0.smithi138.stdout:1486: left oid 45 (ObjNum 712 snap 177 seq_num 712) 2023-06-27T12:44:22.674 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:22.676 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:22.690 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":555,"stamp":"2023-06-27T12:44:22.563518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103367274,"num_objects":371,"num_object_clones":324,"num_object_copies":1113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":2152,"num_read_kb":1581179,"num_write":1955,"num_write_kb":572681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3567,"ondisk_log_size":3567,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1083644,"kb_used_data":947676,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376403716,"statfs":{"total":386547056640,"available":385437405184,"internally_reserved":0,"allocated":970420224,"data_stored":1117153998,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10571900,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":41,"num_read_kb":33365,"num_write":36,"num_write_kb":12122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019996"},"pg_stats":[{"pgid":"3.f","version":"361'54","reported_seq":868,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.563042+0000","last_change":"2023-06-27T12:44:15.568908+0000","last_active":"2023-06-27T12:44:18.563042+0000","last_peered":"2023-06-27T12:44:18.563042+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:18.563042+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3731286,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":13,"num_read_kb":7,"num_write":33,"num_write_kb":9288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"363'234","reported_seq":1150,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.564643+0000","last_change":"2023-06-27T12:44:14.572505+0000","last_active":"2023-06-27T12:44:18.564643+0000","last_peered":"2023-06-27T12:44:18.564643+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:18.564643+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336899,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":204991,"num_write":167,"num_write_kb":50732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"361'285","reported_seq":1123,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.572508+0000","last_change":"2023-06-27T12:44:14.562818+0000","last_active":"2023-06-27T12:44:18.572508+0000","last_peered":"2023-06-27T12:44:18.572508+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:18.572508+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5966716,"num_objects":33,"num_object_clones":28,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":252,"num_read_kb":171532,"num_write":179,"num_write_kb":52451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":33}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"363'198","reported_seq":1073,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.574212+0000","last_change":"2023-06-27T12:44:15.607230+0000","last_active":"2023-06-27T12:44:18.574212+0000","last_peered":"2023-06-27T12:44:18.574212+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:18.574212+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10353508,"num_objects":29,"num_object_clones":25,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":175,"num_read_kb":119557,"num_write":135,"num_write_kb":38638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":29}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":612,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.575732+0000","last_change":"2023-06-27T12:44:15.566352+0000","last_active":"2023-06-27T12:44:18.575732+0000","last_peered":"2023-06-27T12:44:18.575732+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:18.575732+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"358'420","reported_seq":1147,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.574820+0000","last_change":"2023-06-27T12:44:15.649635+0000","last_active":"2023-06-27T12:44:18.574820+0000","last_peered":"2023-06-27T12:44:18.574820+0000","last_clean":"2023-06-27T12:44:18.574820+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:18.574820+0000","last_undegraded":"2023-06-27T12:44:18.574820+0000","last_fullsized":"2023-06-27T12:44:18.574820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6118012,"num_objects":46,"num_object_clones":40,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":313,"num_read_kb":193215,"num_write":243,"num_write_kb":69875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"352'203","reported_seq":960,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.567453+0000","last_change":"2023-06-27T12:44:15.606567+0000","last_active":"2023-06-27T12:44:18.567453+0000","last_peered":"2023-06-27T12:44:18.567453+0000","last_clean":"2023-06-27T12:44:18.567453+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:18.567453+0000","last_undegraded":"2023-06-27T12:44:18.567453+0000","last_fullsized":"2023-06-27T12:44:18.567453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":132,"num_read_kb":85550,"num_write":127,"num_write_kb":37347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":900,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.569695+0000","last_change":"2023-06-27T12:44:14.560605+0000","last_active":"2023-06-27T12:44:18.569695+0000","last_peered":"2023-06-27T12:44:18.569695+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:18.569695+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":612,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.570929+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:18.570929+0000","last_peered":"2023-06-27T12:44:18.570929+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:18.570929+0000","last_undegraded":"2023-06-27T12:44:18.570929+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":808,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.570545+0000","last_change":"2023-06-27T12:44:14.564748+0000","last_active":"2023-06-27T12:44:18.570545+0000","last_peered":"2023-06-27T12:44:18.570545+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:18.570545+0000","last_undegraded":"2023-06-27T12:44:18.570545+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":593,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.573776+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:18.573776+0000","last_peered":"2023-06-27T12:44:18.573776+0000","last_clean":"2023-06-27T12:44:18.573776+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:18.573776+0000","last_undegraded":"2023-06-27T12:44:18.573776+0000","last_fullsized":"2023-06-27T12:44:18.573776+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"349'194","reported_seq":1038,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.563396+0000","last_change":"2023-06-27T12:44:14.552049+0000","last_active":"2023-06-27T12:44:18.563396+0000","last_peered":"2023-06-27T12:44:18.563396+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:18.563396+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6101205,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":80,"num_read_kb":70438,"num_write":124,"num_write_kb":34515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":576,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.559266+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:18.559266+0000","last_peered":"2023-06-27T12:44:18.559266+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:18.559266+0000","last_undegraded":"2023-06-27T12:44:18.559266+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"354'447","reported_seq":719,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.562227+0000","last_change":"2023-06-27T12:44:14.551270+0000","last_active":"2023-06-27T12:44:18.562227+0000","last_peered":"2023-06-27T12:44:18.562227+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:18.562227+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5865907,"num_objects":26,"num_object_clones":24,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":69,"num_read_kb":41842,"num_write":131,"num_write_kb":40677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":573,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.571144+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:18.571144+0000","last_peered":"2023-06-27T12:44:18.571144+0000","last_clean":"2023-06-27T12:44:18.571144+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:18.571144+0000","last_undegraded":"2023-06-27T12:44:18.571144+0000","last_fullsized":"2023-06-27T12:44:18.571144+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":345,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.560082+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:18.560082+0000","last_peered":"2023-06-27T12:44:18.560082+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:18.560082+0000","last_undegraded":"2023-06-27T12:44:18.560082+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"354'401","reported_seq":1254,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.567051+0000","last_change":"2023-06-27T12:44:14.557999+0000","last_active":"2023-06-27T12:44:18.567051+0000","last_peered":"2023-06-27T12:44:18.567051+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:18.567051+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790292,"num_objects":51,"num_object_clones":45,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":319,"num_read_kb":198252,"num_write":283,"num_write_kb":83218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":593,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.566866+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:18.566866+0000","last_peered":"2023-06-27T12:44:18.566866+0000","last_clean":"2023-06-27T12:44:18.566866+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:18.566866+0000","last_undegraded":"2023-06-27T12:44:18.566866+0000","last_fullsized":"2023-06-27T12:44:18.566866+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"354'257","reported_seq":646,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.575821+0000","last_change":"2023-06-27T12:44:14.563669+0000","last_active":"2023-06-27T12:44:18.575821+0000","last_peered":"2023-06-27T12:44:18.575821+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:18.575821+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9866138,"num_objects":19,"num_object_clones":19,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59443,"num_write":105,"num_write_kb":30476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":355,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.562483+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:18.562483+0000","last_peered":"2023-06-27T12:44:18.562483+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:18.562483+0000","last_undegraded":"2023-06-27T12:44:18.562483+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"353'214","reported_seq":1083,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.569381+0000","last_change":"2023-06-27T12:44:14.548447+0000","last_active":"2023-06-27T12:44:18.569381+0000","last_peered":"2023-06-27T12:44:18.569381+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:18.569381+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8588773,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":114,"num_read_kb":123169,"num_write":141,"num_write_kb":40461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"361'144","reported_seq":602,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.569732+0000","last_change":"2023-06-27T12:44:14.558520+0000","last_active":"2023-06-27T12:44:18.569732+0000","last_peered":"2023-06-27T12:44:18.569732+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:18.569732+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3815321,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":101,"num_read_kb":56967,"num_write":58,"num_write_kb":16387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":612,"reported_epoch":364,"state":"active+undersized","last_fresh":"2023-06-27T12:44:18.564741+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:18.564741+0000","last_peered":"2023-06-27T12:44:18.564741+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:18.564741+0000","last_undegraded":"2023-06-27T12:44:18.564741+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":573,"reported_epoch":364,"state":"active+clean","last_fresh":"2023-06-27T12:44:18.572870+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:18.572870+0000","last_peered":"2023-06-27T12:44:18.572870+0000","last_clean":"2023-06-27T12:44:18.572870+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:18.572870+0000","last_undegraded":"2023-06-27T12:44:18.572870+0000","last_fullsized":"2023-06-27T12:44:18.572870+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"361'236","reported_seq":1081,"reported_epoch":364,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:18.561173+0000","last_change":"2023-06-27T12:44:14.548401+0000","last_active":"2023-06-27T12:44:18.561173+0000","last_peered":"2023-06-27T12:44:18.561173+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:18.561173+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5916328,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":122,"num_read_kb":132246,"num_write":125,"num_write_kb":37073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":103367274,"num_objects":371,"num_object_clones":324,"num_object_copies":1113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":2152,"num_read_kb":1581179,"num_write":1955,"num_write_kb":572681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3470,"ondisk_log_size":3470,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673345,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":287048,"kb_used_data":252032,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94084792,"statfs":{"total":96636764160,"available":96342827008,"internally_reserved":0,"allocated":258080768,"data_stored":298525801,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":293764,"kb_used_data":258748,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94078076,"statfs":{"total":96636764160,"available":96335949824,"internally_reserved":0,"allocated":264957952,"data_stored":302705379,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673345,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":321552,"kb_used_data":286536,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94050288,"statfs":{"total":96636764160,"available":96307494912,"internally_reserved":0,"allocated":293412864,"data_stored":338883266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:22.692 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.c 3.b 3.8 3.3 3.1 3.4 2023-06-27T12:44:22.707 INFO:tasks.rados.rados.0.smithi138.stdout:1486: done (4 left) 2023-06-27T12:44:22.708 INFO:tasks.rados.rados.0.smithi138.stdout:1491: done (3 left) 2023-06-27T12:44:22.708 INFO:tasks.rados.rados.0.smithi138.stdout:1492: delete oid 32 current snap is 177 2023-06-27T12:44:22.735 INFO:tasks.rados.rados.0.smithi138.stdout:1488: finishing write tid 1 to smithi138138488-31 2023-06-27T12:44:22.737 INFO:tasks.rados.rados.0.smithi138.stdout:1488: finishing write tid 2 to smithi138138488-31 2023-06-27T12:44:22.740 INFO:tasks.rados.rados.0.smithi138.stdout:1489: expect (ObjNum 679 snap 167 seq_num 679) 2023-06-27T12:44:22.837 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:22.822+0000 7f89da820700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:22.840 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:22.836+0000 7f89da820700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:22.924 INFO:tasks.rados.rados.0.smithi138.stdout:1488: finishing write tid 3 to smithi138138488-31 2023-06-27T12:44:22.924 INFO:tasks.rados.rados.0.smithi138.stdout:1488: finishing write tid 4 to smithi138138488-31 2023-06-27T12:44:22.924 INFO:tasks.rados.rados.0.smithi138.stdout:1489: done (3 left) 2023-06-27T12:44:22.924 INFO:tasks.rados.rados.0.smithi138.stdout:1492: done (2 left) 2023-06-27T12:44:22.924 INFO:tasks.rados.rados.0.smithi138.stdout:1493: snap_remove snap 174 2023-06-27T12:44:23.034 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.c,3.f] on osd.0 to force-backfill; instructing pg(s) [3.1,3.8] on osd.1 to force-backfill; instructing pg(s) [3.3,3.4] on osd.2 to force-backfill; 2023-06-27T12:44:23.034 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:44:23.150 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:23.274 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:23.269+0000 7fbbc8bec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:23.277 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:23.273+0000 7fbbc8bec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:23.323 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:23.326 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:23.327 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:23.572 INFO:tasks.rados.rados.0.smithi138.stdout:1493: done (2 left) 2023-06-27T12:44:23.572 INFO:tasks.rados.rados.0.smithi138.stdout:1494: snap_create 2023-06-27T12:44:23.572 INFO:tasks.rados.rados.0.smithi138.stdout:1488: finishing write tid 5 to smithi138138488-31 2023-06-27T12:44:23.573 INFO:tasks.rados.rados.0.smithi138.stdout:1488: oid 31 updating version 0 to 432 2023-06-27T12:44:23.573 INFO:tasks.rados.rados.0.smithi138.stdout:1488: oid 31 version 432 is already newer than 432 2023-06-27T12:44:23.573 INFO:tasks.rados.rados.0.smithi138.stdout:1488: oid 31 version 432 is already newer than 432 2023-06-27T12:44:23.573 INFO:tasks.rados.rados.0.smithi138.stdout:1488: oid 31 version 432 is already newer than 432 2023-06-27T12:44:23.573 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 432 (ObjNum 713 snap 177 seq_num 713) dirty exists 2023-06-27T12:44:23.573 INFO:tasks.rados.rados.0.smithi138.stdout:1488: left oid 31 (ObjNum 713 snap 177 seq_num 713) 2023-06-27T12:44:24.577 INFO:tasks.rados.rados.0.smithi138.stdout:1488: done (2 left) 2023-06-27T12:44:24.578 INFO:tasks.rados.rados.0.smithi138.stdout:1490: done (1 left) 2023-06-27T12:44:24.578 INFO:tasks.rados.rados.0.smithi138.stdout:1494: done (0 left) 2023-06-27T12:44:24.579 INFO:tasks.rados.rados.0.smithi138.stdout:1495: write (excl) oid 23 current snap is 178 2023-06-27T12:44:24.579 INFO:tasks.rados.rados.0.smithi138.stdout:1495: seq_num 716 ranges {433812=415466,1648519=385293} 2023-06-27T12:44:24.582 INFO:tasks.rados.rados.0.smithi138.stdout:1495: writing smithi138138488-23 from 433812 to 849278 tid 1 2023-06-27T12:44:24.588 INFO:tasks.rados.rados.0.smithi138.stdout:1495: writing smithi138138488-23 from 1648519 to 2033812 tid 2 2023-06-27T12:44:24.588 INFO:tasks.rados.rados.0.smithi138.stdout:1496: read oid 33 snap 172 2023-06-27T12:44:24.589 INFO:tasks.rados.rados.0.smithi138.stdout:1496: expect (ObjNum 660 snap 163 seq_num 660) 2023-06-27T12:44:24.589 INFO:tasks.rados.rados.0.smithi138.stdout:1497: write oid 12 current snap is 178 2023-06-27T12:44:24.589 INFO:tasks.rados.rados.0.smithi138.stdout:1497: seq_num 717 ranges {450619=490715,1590648=459971} 2023-06-27T12:44:24.593 INFO:tasks.rados.rados.0.smithi138.stdout:1497: writing smithi138138488-12 from 450619 to 941334 tid 1 2023-06-27T12:44:24.601 INFO:tasks.rados.rados.0.smithi138.stdout:1497: writing smithi138138488-12 from 1590648 to 2050619 tid 2 2023-06-27T12:44:24.601 INFO:tasks.rados.rados.0.smithi138.stdout:1498: read oid 44 snap -1 2023-06-27T12:44:24.601 INFO:tasks.rados.rados.0.smithi138.stdout:1498: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:44:24.601 INFO:tasks.rados.rados.0.smithi138.stdout:1499: write (excl) oid 5 current snap is 178 2023-06-27T12:44:24.602 INFO:tasks.rados.rados.0.smithi138.stdout:1499: seq_num 718 ranges {467426=565964,1649130=418296} 2023-06-27T12:44:24.607 INFO:tasks.rados.rados.0.smithi138.stdout:1499: writing smithi138138488-5 from 467426 to 1033390 tid 1 2023-06-27T12:44:24.613 INFO:tasks.rados.rados.0.smithi138.stdout:1499: writing smithi138138488-5 from 1649130 to 2067426 tid 2 2023-06-27T12:44:24.613 INFO:tasks.rados.rados.0.smithi138.stdout:1500: snap_remove snap 167 2023-06-27T12:44:25.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:25.457 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:25.452+0000 7f5860907700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:25.460 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:25.456+0000 7f5860907700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:25.506 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:25.508 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:25.509 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:25.581 INFO:tasks.rados.rados.0.smithi138.stdout:1500: done (5 left) 2023-06-27T12:44:25.581 INFO:tasks.rados.rados.0.smithi138.stdout:1501: delete oid 1 current snap is 178 2023-06-27T12:44:25.581 INFO:tasks.rados.rados.0.smithi138.stdout:1495: finishing write tid 1 to smithi138138488-23 2023-06-27T12:44:25.582 INFO:tasks.rados.rados.0.smithi138.stdout:1498: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:44:25.796 INFO:tasks.rados.rados.0.smithi138.stdout:1496: expect (ObjNum 660 snap 163 seq_num 660) 2023-06-27T12:44:25.996 INFO:tasks.rados.rados.0.smithi138.stdout:1495: finishing write tid 2 to smithi138138488-23 2023-06-27T12:44:25.997 INFO:tasks.rados.rados.0.smithi138.stdout:1495: finishing write tid 3 to smithi138138488-23 2023-06-27T12:44:25.997 INFO:tasks.rados.rados.0.smithi138.stdout:1495: finishing write tid 4 to smithi138138488-23 2023-06-27T12:44:25.997 INFO:tasks.rados.rados.0.smithi138.stdout:1495: oid 23 updating version 0 to 201 2023-06-27T12:44:25.997 INFO:tasks.rados.rados.0.smithi138.stdout:1495: oid 23 version 201 is already newer than 201 2023-06-27T12:44:25.997 INFO:tasks.rados.rados.0.smithi138.stdout:1495: oid 23 version 201 is already newer than 199 2023-06-27T12:44:25.998 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 201 (ObjNum 716 snap 178 seq_num 716) dirty exists 2023-06-27T12:44:25.998 INFO:tasks.rados.rados.0.smithi138.stdout:1495: left oid 23 (ObjNum 716 snap 178 seq_num 716) 2023-06-27T12:44:25.998 INFO:tasks.rados.rados.0.smithi138.stdout:1497: finishing write tid 1 to smithi138138488-12 2023-06-27T12:44:25.998 INFO:tasks.rados.rados.0.smithi138.stdout:1497: finishing write tid 2 to smithi138138488-12 2023-06-27T12:44:25.998 INFO:tasks.rados.rados.0.smithi138.stdout:1495: done (5 left) 2023-06-27T12:44:25.998 INFO:tasks.rados.rados.0.smithi138.stdout:1496: done (4 left) 2023-06-27T12:44:25.998 INFO:tasks.rados.rados.0.smithi138.stdout:1498: done (3 left) 2023-06-27T12:44:25.998 INFO:tasks.rados.rados.0.smithi138.stdout:1501: done (2 left) 2023-06-27T12:44:25.999 INFO:tasks.rados.rados.0.smithi138.stdout:1502: read oid 45 snap -1 2023-06-27T12:44:25.999 INFO:tasks.rados.rados.0.smithi138.stdout:1502: expect (ObjNum 712 snap 177 seq_num 712) 2023-06-27T12:44:25.999 INFO:tasks.rados.rados.0.smithi138.stdout:1503: write (excl) oid 10 current snap is 178 2023-06-27T12:44:25.999 INFO:tasks.rados.rados.0.smithi138.stdout:1499: finishing write tid 1 to smithi138138488-5 2023-06-27T12:44:25.999 INFO:tasks.rados.rados.0.smithi138.stdout:1503: seq_num 720 ranges {501040=716462,1766094=334946} 2023-06-27T12:44:26.006 INFO:tasks.rados.rados.0.smithi138.stdout:1503: writing smithi138138488-10 from 501040 to 1217502 tid 1 2023-06-27T12:44:26.013 INFO:tasks.rados.rados.0.smithi138.stdout:1503: writing smithi138138488-10 from 1766094 to 2101040 tid 2 2023-06-27T12:44:26.013 INFO:tasks.rados.rados.0.smithi138.stdout:1504: delete oid 42 current snap is 178 2023-06-27T12:44:26.013 INFO:tasks.rados.rados.0.smithi138.stdout:1497: finishing write tid 3 to smithi138138488-12 2023-06-27T12:44:26.013 INFO:tasks.rados.rados.0.smithi138.stdout:1497: finishing write tid 4 to smithi138138488-12 2023-06-27T12:44:26.014 INFO:tasks.rados.rados.0.smithi138.stdout:1497: oid 12 updating version 0 to 298 2023-06-27T12:44:26.014 INFO:tasks.rados.rados.0.smithi138.stdout:1497: oid 12 version 298 is already newer than 298 2023-06-27T12:44:26.014 INFO:tasks.rados.rados.0.smithi138.stdout:1497: oid 12 version 298 is already newer than 298 2023-06-27T12:44:26.014 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 298 (ObjNum 717 snap 178 seq_num 717) dirty exists 2023-06-27T12:44:26.014 INFO:tasks.rados.rados.0.smithi138.stdout:1497: left oid 12 (ObjNum 717 snap 178 seq_num 717) 2023-06-27T12:44:26.014 INFO:tasks.rados.rados.0.smithi138.stdout:1499: finishing write tid 2 to smithi138138488-5 2023-06-27T12:44:26.014 INFO:tasks.rados.rados.0.smithi138.stdout:1499: finishing write tid 3 to smithi138138488-5 2023-06-27T12:44:26.014 INFO:tasks.rados.rados.0.smithi138.stdout:1499: finishing write tid 4 to smithi138138488-5 2023-06-27T12:44:26.015 INFO:tasks.rados.rados.0.smithi138.stdout:1499: oid 5 updating version 0 to 261 2023-06-27T12:44:26.015 INFO:tasks.rados.rados.0.smithi138.stdout:1499: oid 5 version 261 is already newer than 261 2023-06-27T12:44:26.015 INFO:tasks.rados.rados.0.smithi138.stdout:1499: oid 5 version 261 is already newer than 258 2023-06-27T12:44:26.015 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 5 v 261 (ObjNum 718 snap 178 seq_num 718) dirty exists 2023-06-27T12:44:26.015 INFO:tasks.rados.rados.0.smithi138.stdout:1499: left oid 5 (ObjNum 718 snap 178 seq_num 718) 2023-06-27T12:44:26.057 INFO:tasks.rados.rados.0.smithi138.stdout:1503: finishing write tid 1 to smithi138138488-10 2023-06-27T12:44:26.060 INFO:tasks.rados.rados.0.smithi138.stdout:1502: expect (ObjNum 712 snap 177 seq_num 712) 2023-06-27T12:44:26.212 INFO:tasks.rados.rados.0.smithi138.stdout:1503: finishing write tid 2 to smithi138138488-10 2023-06-27T12:44:26.212 INFO:tasks.rados.rados.0.smithi138.stdout:1503: finishing write tid 3 to smithi138138488-10 2023-06-27T12:44:26.213 INFO:tasks.rados.rados.0.smithi138.stdout:1503: finishing write tid 4 to smithi138138488-10 2023-06-27T12:44:26.213 INFO:tasks.rados.rados.0.smithi138.stdout:1503: oid 10 updating version 0 to 210 2023-06-27T12:44:26.213 INFO:tasks.rados.rados.0.smithi138.stdout:1503: oid 10 updating version 210 to 213 2023-06-27T12:44:26.213 INFO:tasks.rados.rados.0.smithi138.stdout:1503: oid 10 version 213 is already newer than 213 2023-06-27T12:44:26.213 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 213 (ObjNum 720 snap 178 seq_num 720) dirty exists 2023-06-27T12:44:26.213 INFO:tasks.rados.rados.0.smithi138.stdout:1503: left oid 10 (ObjNum 720 snap 178 seq_num 720) 2023-06-27T12:44:26.213 INFO:tasks.rados.rados.0.smithi138.stdout:1497: done (4 left) 2023-06-27T12:44:26.213 INFO:tasks.rados.rados.0.smithi138.stdout:1499: done (3 left) 2023-06-27T12:44:26.214 INFO:tasks.rados.rados.0.smithi138.stdout:1502: done (2 left) 2023-06-27T12:44:26.214 INFO:tasks.rados.rados.0.smithi138.stdout:1503: done (1 left) 2023-06-27T12:44:26.214 INFO:tasks.rados.rados.0.smithi138.stdout:1504: done (0 left) 2023-06-27T12:44:26.214 INFO:tasks.rados.rados.0.smithi138.stdout:1505: read oid 44 snap -1 2023-06-27T12:44:26.214 INFO:tasks.rados.rados.0.smithi138.stdout:1505: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:44:26.214 INFO:tasks.rados.rados.0.smithi138.stdout:1506: read oid 42 snap 175 2023-06-27T12:44:26.214 INFO:tasks.rados.rados.0.smithi138.stdout:1506: expect (ObjNum 705 snap 173 seq_num 705) 2023-06-27T12:44:26.214 INFO:tasks.rados.rados.0.smithi138.stdout:1507: write oid 28 current snap is 178 2023-06-27T12:44:26.215 INFO:tasks.rados.rados.0.smithi138.stdout:1507: seq_num 722 ranges {534654=466960,1766705=367949} 2023-06-27T12:44:26.220 INFO:tasks.rados.rados.0.smithi138.stdout:1507: writing smithi138138488-28 from 534654 to 1001614 tid 1 2023-06-27T12:44:26.228 INFO:tasks.rados.rados.0.smithi138.stdout:1507: writing smithi138138488-28 from 1766705 to 2134654 tid 2 2023-06-27T12:44:26.228 INFO:tasks.rados.rados.0.smithi138.stdout:1508: write oid 7 current snap is 178 2023-06-27T12:44:26.228 INFO:tasks.rados.rados.0.smithi138.stdout:1508: seq_num 723 ranges {551461=658562,1941540=209921} 2023-06-27T12:44:26.238 INFO:tasks.rados.rados.0.smithi138.stdout:1508: writing smithi138138488-7 from 551461 to 1210023 tid 1 2023-06-27T12:44:26.244 INFO:tasks.rados.rados.0.smithi138.stdout:1508: writing smithi138138488-7 from 1941540 to 2151461 tid 2 2023-06-27T12:44:26.245 INFO:tasks.rados.rados.0.smithi138.stdout:1509: read oid 26 snap 170 2023-06-27T12:44:26.245 INFO:tasks.rados.rados.0.smithi138.stdout:1509: expect deleted 2023-06-27T12:44:26.245 INFO:tasks.rados.rados.0.smithi138.stdout:1510: write (excl) oid 34 current snap is 178 2023-06-27T12:44:26.245 INFO:tasks.rados.rados.0.smithi138.stdout:1510: seq_num 724 ranges {568268=733811,2000022=168246} 2023-06-27T12:44:26.254 INFO:tasks.rados.rados.0.smithi138.stdout:1510: writing smithi138138488-34 from 568268 to 1302079 tid 1 2023-06-27T12:44:26.260 INFO:tasks.rados.rados.0.smithi138.stdout:1510: writing smithi138138488-34 from 2000022 to 2168268 tid 2 2023-06-27T12:44:26.260 INFO:tasks.rados.rados.0.smithi138.stdout:1511: snap_remove snap 165 2023-06-27T12:44:26.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~231s 2023-06-27T12:44:26.586 INFO:tasks.rados.rados.0.smithi138.stdout:1511: done (6 left) 2023-06-27T12:44:26.587 INFO:tasks.rados.rados.0.smithi138.stdout:1512: write oid 5 current snap is 178 2023-06-27T12:44:26.587 INFO:tasks.rados.rados.0.smithi138.stdout:1505: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:44:26.781 INFO:tasks.rados.rados.0.smithi138.stdout:1506: expect (ObjNum 705 snap 173 seq_num 705) 2023-06-27T12:44:27.073 INFO:tasks.rados.rados.0.smithi138.stdout:1507: finishing write tid 1 to smithi138138488-28 2023-06-27T12:44:27.074 INFO:tasks.rados.rados.0.smithi138.stdout:1507: finishing write tid 2 to smithi138138488-28 2023-06-27T12:44:27.074 INFO:tasks.rados.rados.0.smithi138.stdout:1507: finishing write tid 3 to smithi138138488-28 2023-06-27T12:44:27.074 INFO:tasks.rados.rados.0.smithi138.stdout:1507: finishing write tid 4 to smithi138138488-28 2023-06-27T12:44:27.074 INFO:tasks.rados.rados.0.smithi138.stdout:1507: oid 28 updating version 0 to 216 2023-06-27T12:44:27.074 INFO:tasks.rados.rados.0.smithi138.stdout:1507: oid 28 version 216 is already newer than 216 2023-06-27T12:44:27.074 INFO:tasks.rados.rados.0.smithi138.stdout:1507: oid 28 version 216 is already newer than 216 2023-06-27T12:44:27.075 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 216 (ObjNum 722 snap 178 seq_num 722) dirty exists 2023-06-27T12:44:27.075 INFO:tasks.rados.rados.0.smithi138.stdout:1507: left oid 28 (ObjNum 722 snap 178 seq_num 722) 2023-06-27T12:44:27.075 INFO:tasks.rados.rados.0.smithi138.stdout:1510: finishing write tid 1 to smithi138138488-34 2023-06-27T12:44:27.075 INFO:tasks.rados.rados.0.smithi138.stdout:1512: seq_num 725 ranges {585075=409060,1542151=642924} 2023-06-27T12:44:27.079 INFO:tasks.rados.rados.0.smithi138.stdout:1512: writing smithi138138488-5 from 585075 to 994135 tid 1 2023-06-27T12:44:27.089 INFO:tasks.rados.rados.0.smithi138.stdout:1512: writing smithi138138488-5 from 1542151 to 2185075 tid 2 2023-06-27T12:44:27.089 INFO:tasks.rados.rados.0.smithi138.stdout:1505: done (6 left) 2023-06-27T12:44:27.089 INFO:tasks.rados.rados.0.smithi138.stdout:1506: done (5 left) 2023-06-27T12:44:27.089 INFO:tasks.rados.rados.0.smithi138.stdout:1507: done (4 left) 2023-06-27T12:44:27.089 INFO:tasks.rados.rados.0.smithi138.stdout:1509: done (3 left) 2023-06-27T12:44:27.089 INFO:tasks.rados.rados.0.smithi138.stdout:1513: read oid 48 snap -1 2023-06-27T12:44:27.090 INFO:tasks.rados.rados.0.smithi138.stdout:1513: expect (ObjNum 602 snap 144 seq_num 602) 2023-06-27T12:44:27.090 INFO:tasks.rados.rados.0.smithi138.stdout:1514: write oid 20 current snap is 178 2023-06-27T12:44:27.090 INFO:tasks.rados.rados.0.smithi138.stdout:1508: finishing write tid 1 to smithi138138488-7 2023-06-27T12:44:27.090 INFO:tasks.rados.rados.0.smithi138.stdout:1514: seq_num 726 ranges {601882=484309,1600633=524904} 2023-06-27T12:44:27.096 INFO:tasks.rados.rados.0.smithi138.stdout:1514: writing smithi138138488-20 from 601882 to 1086191 tid 1 2023-06-27T12:44:27.105 INFO:tasks.rados.rados.0.smithi138.stdout:1514: writing smithi138138488-20 from 1600633 to 2125537 tid 2 2023-06-27T12:44:27.105 INFO:tasks.rados.rados.0.smithi138.stdout:1515: delete oid 1 current snap is 178 2023-06-27T12:44:27.105 INFO:tasks.rados.rados.0.smithi138.stdout:1508: finishing write tid 2 to smithi138138488-7 2023-06-27T12:44:27.105 INFO:tasks.rados.rados.0.smithi138.stdout:1508: finishing write tid 3 to smithi138138488-7 2023-06-27T12:44:27.106 INFO:tasks.rados.rados.0.smithi138.stdout:1508: finishing write tid 4 to smithi138138488-7 2023-06-27T12:44:27.106 INFO:tasks.rados.rados.0.smithi138.stdout:1508: oid 7 updating version 0 to 220 2023-06-27T12:44:27.106 INFO:tasks.rados.rados.0.smithi138.stdout:1508: oid 7 version 220 is already newer than 220 2023-06-27T12:44:27.106 INFO:tasks.rados.rados.0.smithi138.stdout:1508: oid 7 version 220 is already newer than 220 2023-06-27T12:44:27.106 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 220 (ObjNum 723 snap 178 seq_num 723) dirty exists 2023-06-27T12:44:27.106 INFO:tasks.rados.rados.0.smithi138.stdout:1508: left oid 7 (ObjNum 723 snap 178 seq_num 723) 2023-06-27T12:44:27.106 INFO:tasks.rados.rados.0.smithi138.stdout:1510: finishing write tid 2 to smithi138138488-34 2023-06-27T12:44:27.106 INFO:tasks.rados.rados.0.smithi138.stdout:1510: finishing write tid 3 to smithi138138488-34 2023-06-27T12:44:27.107 INFO:tasks.rados.rados.0.smithi138.stdout:1510: finishing write tid 4 to smithi138138488-34 2023-06-27T12:44:27.107 INFO:tasks.rados.rados.0.smithi138.stdout:1510: oid 34 updating version 0 to 439 2023-06-27T12:44:27.107 INFO:tasks.rados.rados.0.smithi138.stdout:1510: oid 34 version 439 is already newer than 439 2023-06-27T12:44:27.107 INFO:tasks.rados.rados.0.smithi138.stdout:1510: oid 34 version 439 is already newer than 437 2023-06-27T12:44:27.107 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 439 (ObjNum 724 snap 178 seq_num 724) dirty exists 2023-06-27T12:44:27.107 INFO:tasks.rados.rados.0.smithi138.stdout:1510: left oid 34 (ObjNum 724 snap 178 seq_num 724) 2023-06-27T12:44:27.168 INFO:tasks.rados.rados.0.smithi138.stdout:1514: finishing write tid 1 to smithi138138488-20 2023-06-27T12:44:27.184 INFO:tasks.rados.rados.0.smithi138.stdout:1512: finishing write tid 1 to smithi138138488-5 2023-06-27T12:44:27.189 INFO:tasks.rados.rados.0.smithi138.stdout:1508: done (5 left) 2023-06-27T12:44:27.189 INFO:tasks.rados.rados.0.smithi138.stdout:1510: done (4 left) 2023-06-27T12:44:27.189 INFO:tasks.rados.rados.0.smithi138.stdout:1515: done (3 left) 2023-06-27T12:44:27.190 INFO:tasks.rados.rados.0.smithi138.stdout:1516: write oid 14 current snap is 178 2023-06-27T12:44:27.190 INFO:tasks.rados.rados.0.smithi138.stdout:1516: seq_num 728 ranges {635496=634807,1717597=517899} 2023-06-27T12:44:27.198 INFO:tasks.rados.rados.0.smithi138.stdout:1516: writing smithi138138488-14 from 635496 to 1270303 tid 1 2023-06-27T12:44:27.205 INFO:tasks.rados.rados.0.smithi138.stdout:1516: writing smithi138138488-14 from 1717597 to 2235496 tid 2 2023-06-27T12:44:27.206 INFO:tasks.rados.rados.0.smithi138.stdout:1517: delete oid 2 current snap is 178 2023-06-27T12:44:27.206 INFO:tasks.rados.rados.0.smithi138.stdout:1513: expect (ObjNum 602 snap 144 seq_num 602) 2023-06-27T12:44:27.341 INFO:tasks.rados.rados.0.smithi138.stdout:1512: finishing write tid 2 to smithi138138488-5 2023-06-27T12:44:27.341 INFO:tasks.rados.rados.0.smithi138.stdout:1512: finishing write tid 3 to smithi138138488-5 2023-06-27T12:44:27.341 INFO:tasks.rados.rados.0.smithi138.stdout:1512: finishing write tid 4 to smithi138138488-5 2023-06-27T12:44:27.341 INFO:tasks.rados.rados.0.smithi138.stdout:1512: oid 5 updating version 0 to 266 2023-06-27T12:44:27.341 INFO:tasks.rados.rados.0.smithi138.stdout:1512: oid 5 version 266 is already newer than 266 2023-06-27T12:44:27.341 INFO:tasks.rados.rados.0.smithi138.stdout:1512: oid 5 version 266 is already newer than 266 2023-06-27T12:44:27.342 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 5 v 266 (ObjNum 725 snap 178 seq_num 725) dirty exists 2023-06-27T12:44:27.342 INFO:tasks.rados.rados.0.smithi138.stdout:1512: left oid 5 (ObjNum 725 snap 178 seq_num 725) 2023-06-27T12:44:27.342 INFO:tasks.rados.rados.0.smithi138.stdout:1514: finishing write tid 2 to smithi138138488-20 2023-06-27T12:44:27.342 INFO:tasks.rados.rados.0.smithi138.stdout:1516: finishing write tid 1 to smithi138138488-14 2023-06-27T12:44:27.342 INFO:tasks.rados.rados.0.smithi138.stdout:1514: finishing write tid 3 to smithi138138488-20 2023-06-27T12:44:27.343 INFO:tasks.rados.rados.0.smithi138.stdout:1514: finishing write tid 4 to smithi138138488-20 2023-06-27T12:44:27.343 INFO:tasks.rados.rados.0.smithi138.stdout:1514: oid 20 updating version 0 to 249 2023-06-27T12:44:27.343 INFO:tasks.rados.rados.0.smithi138.stdout:1514: oid 20 version 249 is already newer than 249 2023-06-27T12:44:27.344 INFO:tasks.rados.rados.0.smithi138.stdout:1514: oid 20 version 249 is already newer than 249 2023-06-27T12:44:27.344 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 249 (ObjNum 726 snap 178 seq_num 726) dirty exists 2023-06-27T12:44:27.344 INFO:tasks.rados.rados.0.smithi138.stdout:1514: left oid 20 (ObjNum 726 snap 178 seq_num 726) 2023-06-27T12:44:27.345 INFO:tasks.rados.rados.0.smithi138.stdout:1516: finishing write tid 2 to smithi138138488-14 2023-06-27T12:44:27.345 INFO:tasks.rados.rados.0.smithi138.stdout:1512: done (4 left) 2023-06-27T12:44:27.345 INFO:tasks.rados.rados.0.smithi138.stdout:1513: done (3 left) 2023-06-27T12:44:27.345 INFO:tasks.rados.rados.0.smithi138.stdout:1514: done (2 left) 2023-06-27T12:44:27.346 INFO:tasks.rados.rados.0.smithi138.stdout:1517: done (1 left) 2023-06-27T12:44:27.346 INFO:tasks.rados.rados.0.smithi138.stdout:1518: read oid 4 snap -1 2023-06-27T12:44:27.346 INFO:tasks.rados.rados.0.smithi138.stdout:1518: expect (ObjNum 682 snap 167 seq_num 682) 2023-06-27T12:44:27.346 INFO:tasks.rados.rados.0.smithi138.stdout:1519: snap_remove snap 171 2023-06-27T12:44:27.510 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:27.591 INFO:tasks.rados.rados.0.smithi138.stdout:1519: done (2 left) 2023-06-27T12:44:27.592 INFO:tasks.rados.rados.0.smithi138.stdout:1520: read oid 43 snap -1 2023-06-27T12:44:27.592 INFO:tasks.rados.rados.0.smithi138.stdout:1520: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:44:27.592 INFO:tasks.rados.rados.0.smithi138.stdout:1521: read oid 18 snap -1 2023-06-27T12:44:27.592 INFO:tasks.rados.rados.0.smithi138.stdout:1521: expect deleted 2023-06-27T12:44:27.592 INFO:tasks.rados.rados.0.smithi138.stdout:1522: write oid 16 current snap is 178 2023-06-27T12:44:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:1516: finishing write tid 3 to smithi138138488-14 2023-06-27T12:44:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:1516: finishing write tid 4 to smithi138138488-14 2023-06-27T12:44:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:1516: oid 14 updating version 0 to 423 2023-06-27T12:44:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:1516: oid 14 version 423 is already newer than 423 2023-06-27T12:44:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:1516: oid 14 version 423 is already newer than 423 2023-06-27T12:44:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 423 (ObjNum 728 snap 178 seq_num 728) dirty exists 2023-06-27T12:44:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:1516: left oid 14 (ObjNum 728 snap 178 seq_num 728) 2023-06-27T12:44:27.593 INFO:tasks.rados.rados.0.smithi138.stdout:1522: seq_num 730 ranges {669110=501658,1834561=434549} 2023-06-27T12:44:27.600 INFO:tasks.rados.rados.0.smithi138.stdout:1522: writing smithi138138488-16 from 669110 to 1170768 tid 1 2023-06-27T12:44:27.608 INFO:tasks.rados.rados.0.smithi138.stdout:1522: writing smithi138138488-16 from 1834561 to 2269110 tid 2 2023-06-27T12:44:27.609 INFO:tasks.rados.rados.0.smithi138.stdout:1516: done (4 left) 2023-06-27T12:44:27.609 INFO:tasks.rados.rados.0.smithi138.stdout:1523: write (excl) oid 5 current snap is 178 2023-06-27T12:44:27.609 INFO:tasks.rados.rados.0.smithi138.stdout:1523: seq_num 731 ranges {685917=576907,1893043=392874} 2023-06-27T12:44:27.618 INFO:tasks.rados.rados.0.smithi138.stdout:1523: writing smithi138138488-5 from 685917 to 1262824 tid 1 2023-06-27T12:44:27.625 INFO:tasks.rados.rados.0.smithi138.stdout:1523: writing smithi138138488-5 from 1893043 to 2285917 tid 2 2023-06-27T12:44:27.626 INFO:tasks.rados.rados.0.smithi138.stdout:1524: write (excl) oid 26 current snap is 178 2023-06-27T12:44:27.626 INFO:tasks.rados.rados.0.smithi138.stdout:1524: seq_num 732 ranges {702724=652156,1951525=351199} 2023-06-27T12:44:27.635 INFO:tasks.rados.rados.0.smithi138.stdout:1524: writing smithi138138488-26 from 702724 to 1354880 tid 1 2023-06-27T12:44:27.642 INFO:tasks.rados.rados.0.smithi138.stdout:1524: writing smithi138138488-26 from 1951525 to 2302724 tid 2 2023-06-27T12:44:27.642 INFO:tasks.rados.rados.0.smithi138.stdout:1525: snap_create 2023-06-27T12:44:27.642 INFO:tasks.rados.rados.0.smithi138.stdout:1518: expect (ObjNum 682 snap 167 seq_num 682) 2023-06-27T12:44:27.662 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:27.656+0000 7f1d14826700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:27.667 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:27.663+0000 7f1d14826700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:27.798 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:27.802 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:27.802 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:27.894 INFO:tasks.rados.rados.0.smithi138.stdout:1520: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:44:28.045 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:28.046 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:28.046 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:28.054 INFO:tasks.rados.rados.0.smithi138.stdout:1522: finishing write tid 1 to smithi138138488-16 2023-06-27T12:44:28.054 INFO:tasks.rados.rados.0.smithi138.stdout:1522: finishing write tid 2 to smithi138138488-16 2023-06-27T12:44:28.054 INFO:tasks.rados.rados.0.smithi138.stdout:1522: finishing write tid 3 to smithi138138488-16 2023-06-27T12:44:28.054 INFO:tasks.rados.rados.0.smithi138.stdout:1522: finishing write tid 4 to smithi138138488-16 2023-06-27T12:44:28.054 INFO:tasks.rados.rados.0.smithi138.stdout:1522: oid 16 updating version 0 to 445 2023-06-27T12:44:28.055 INFO:tasks.rados.rados.0.smithi138.stdout:1522: oid 16 version 445 is already newer than 445 2023-06-27T12:44:28.055 INFO:tasks.rados.rados.0.smithi138.stdout:1522: oid 16 version 445 is already newer than 445 2023-06-27T12:44:28.055 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 16 v 445 (ObjNum 730 snap 178 seq_num 730) dirty exists 2023-06-27T12:44:28.055 INFO:tasks.rados.rados.0.smithi138.stdout:1522: left oid 16 (ObjNum 730 snap 178 seq_num 730) 2023-06-27T12:44:28.055 INFO:tasks.rados.rados.0.smithi138.stdout:1524: finishing write tid 1 to smithi138138488-26 2023-06-27T12:44:28.055 INFO:tasks.rados.rados.0.smithi138.stdout:1524: finishing write tid 2 to smithi138138488-26 2023-06-27T12:44:28.055 INFO:tasks.rados.rados.0.smithi138.stdout:1524: finishing write tid 3 to smithi138138488-26 2023-06-27T12:44:28.056 INFO:tasks.rados.rados.0.smithi138.stdout:1524: finishing write tid 4 to smithi138138488-26 2023-06-27T12:44:28.056 INFO:tasks.rados.rados.0.smithi138.stdout:1524: oid 26 updating version 0 to 232 2023-06-27T12:44:28.056 INFO:tasks.rados.rados.0.smithi138.stdout:1524: oid 26 version 232 is already newer than 232 2023-06-27T12:44:28.056 INFO:tasks.rados.rados.0.smithi138.stdout:1524: oid 26 version 232 is already newer than 230 2023-06-27T12:44:28.056 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 232 (ObjNum 732 snap 178 seq_num 732) dirty exists 2023-06-27T12:44:28.056 INFO:tasks.rados.rados.0.smithi138.stdout:1524: left oid 26 (ObjNum 732 snap 178 seq_num 732) 2023-06-27T12:44:28.056 INFO:tasks.rados.rados.0.smithi138.stdout:1523: finishing write tid 1 to smithi138138488-5 2023-06-27T12:44:28.056 INFO:tasks.rados.rados.0.smithi138.stdout:1523: finishing write tid 2 to smithi138138488-5 2023-06-27T12:44:28.057 INFO:tasks.rados.rados.0.smithi138.stdout:1523: finishing write tid 3 to smithi138138488-5 2023-06-27T12:44:28.057 INFO:tasks.rados.rados.0.smithi138.stdout:1523: finishing write tid 4 to smithi138138488-5 2023-06-27T12:44:28.057 INFO:tasks.rados.rados.0.smithi138.stdout:1523: oid 5 updating version 0 to 269 2023-06-27T12:44:28.057 INFO:tasks.rados.rados.0.smithi138.stdout:1523: oid 5 updating version 269 to 271 2023-06-27T12:44:28.057 INFO:tasks.rados.rados.0.smithi138.stdout:1523: oid 5 version 271 is already newer than 271 2023-06-27T12:44:28.057 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 5 v 271 (ObjNum 731 snap 178 seq_num 731) dirty exists 2023-06-27T12:44:28.057 INFO:tasks.rados.rados.0.smithi138.stdout:1523: left oid 5 (ObjNum 731 snap 178 seq_num 731) 2023-06-27T12:44:28.172 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:28.165+0000 7fa18b5c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:28.171+0000 7fa18b5c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:28.367 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:28.369 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:28.380 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":563,"stamp":"2023-06-27T12:44:27.997125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":115821525,"num_objects":376,"num_object_clones":329,"num_object_copies":1128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":2170,"num_read_kb":1587885,"num_write":1990,"num_write_kb":584473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3607,"ondisk_log_size":3607,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1110872,"kb_used_data":974904,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376376488,"statfs":{"total":386547056640,"available":385409523712,"internally_reserved":0,"allocated":998301696,"data_stored":1149000409,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12454251,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":18,"num_read_kb":6706,"num_write":35,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001097"},"pg_stats":[{"pgid":"3.f","version":"361'54","reported_seq":875,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:22.600738+0000","last_change":"2023-06-27T12:44:22.620125+0000","last_active":"2023-06-27T12:44:22.600738+0000","last_peered":"2023-06-27T12:44:22.600738+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:22.600738+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3731286,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":13,"num_read_kb":7,"num_write":33,"num_write_kb":9288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"365'239","reported_seq":1167,"reported_epoch":369,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:23.590474+0000","last_change":"2023-06-27T12:44:23.653439+0000","last_active":"2023-06-27T12:44:23.590474+0000","last_peered":"2023-06-27T12:44:23.590474+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:23.590474+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15177249,"num_objects":31,"num_object_clones":27,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":204992,"num_write":172,"num_write_kb":52600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":31}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"366'289","reported_seq":1138,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:22.597445+0000","last_change":"2023-06-27T12:44:22.730816+0000","last_active":"2023-06-27T12:44:22.597445+0000","last_peered":"2023-06-27T12:44:22.597445+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:22.597445+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9866072,"num_objects":33,"num_object_clones":28,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":254,"num_read_kb":171533,"num_write":184,"num_write_kb":53937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":33}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"363'198","reported_seq":1083,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:22.733438+0000","last_change":"2023-06-27T12:44:22.733438+0000","last_active":"2023-06-27T12:44:22.733438+0000","last_peered":"2023-06-27T12:44:22.733438+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:22.733438+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10353508,"num_objects":29,"num_object_clones":25,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":182,"num_read_kb":126256,"num_write":135,"num_write_kb":38638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":29}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":619,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:22.584767+0000","last_change":"2023-06-27T12:44:22.601118+0000","last_active":"2023-06-27T12:44:22.584767+0000","last_peered":"2023-06-27T12:44:22.584767+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:22.584767+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"368'432","reported_seq":1164,"reported_epoch":368,"state":"active+clean","last_fresh":"2023-06-27T12:44:22.669348+0000","last_change":"2023-06-27T12:44:22.600799+0000","last_active":"2023-06-27T12:44:22.669348+0000","last_peered":"2023-06-27T12:44:22.669348+0000","last_clean":"2023-06-27T12:44:22.669348+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:22.669348+0000","last_undegraded":"2023-06-27T12:44:22.669348+0000","last_fullsized":"2023-06-27T12:44:22.669348+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11597226,"num_objects":47,"num_object_clones":41,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":315,"num_read_kb":193217,"num_write":253,"num_write_kb":73232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"352'203","reported_seq":967,"reported_epoch":368,"state":"active+clean","last_fresh":"2023-06-27T12:44:22.602836+0000","last_change":"2023-06-27T12:44:22.626487+0000","last_active":"2023-06-27T12:44:22.602836+0000","last_peered":"2023-06-27T12:44:22.602836+0000","last_clean":"2023-06-27T12:44:22.602836+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:22.602836+0000","last_undegraded":"2023-06-27T12:44:22.602836+0000","last_fullsized":"2023-06-27T12:44:22.602836+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":25,"num_object_clones":22,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":132,"num_read_kb":85550,"num_write":127,"num_write_kb":37347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":909,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:22.598935+0000","last_change":"2023-06-27T12:44:22.609944+0000","last_active":"2023-06-27T12:44:22.598935+0000","last_peered":"2023-06-27T12:44:22.598935+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:22.598935+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":622,"reported_epoch":369,"state":"active+undersized","last_fresh":"2023-06-27T12:44:23.582790+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:23.582790+0000","last_peered":"2023-06-27T12:44:23.582790+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:23.582790+0000","last_undegraded":"2023-06-27T12:44:23.582790+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":817,"reported_epoch":368,"state":"active+undersized","last_fresh":"2023-06-27T12:44:22.601900+0000","last_change":"2023-06-27T12:44:22.613564+0000","last_active":"2023-06-27T12:44:22.601900+0000","last_peered":"2023-06-27T12:44:22.601900+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:22.601900+0000","last_undegraded":"2023-06-27T12:44:22.601900+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":599,"reported_epoch":368,"state":"active+clean","last_fresh":"2023-06-27T12:44:22.615036+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:22.615036+0000","last_peered":"2023-06-27T12:44:22.615036+0000","last_clean":"2023-06-27T12:44:22.615036+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:22.615036+0000","last_undegraded":"2023-06-27T12:44:22.615036+0000","last_fullsized":"2023-06-27T12:44:22.615036+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"349'194","reported_seq":1050,"reported_epoch":369,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:23.587233+0000","last_change":"2023-06-27T12:44:23.598497+0000","last_active":"2023-06-27T12:44:23.587233+0000","last_peered":"2023-06-27T12:44:23.587233+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:23.587233+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6101205,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":80,"num_read_kb":70438,"num_write":124,"num_write_kb":34515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":584,"reported_epoch":368,"state":"active+undersized","last_fresh":"2023-06-27T12:44:22.600276+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:22.600276+0000","last_peered":"2023-06-27T12:44:22.600276+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:22.600276+0000","last_undegraded":"2023-06-27T12:44:22.600276+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"368'453","reported_seq":737,"reported_epoch":369,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:23.592317+0000","last_change":"2023-06-27T12:44:23.652688+0000","last_active":"2023-06-27T12:44:23.592317+0000","last_peered":"2023-06-27T12:44:23.592317+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:23.592317+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6286082,"num_objects":27,"num_object_clones":25,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":72,"num_read_kb":41843,"num_write":136,"num_write_kb":42509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":27}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":581,"reported_epoch":368,"state":"active+clean","last_fresh":"2023-06-27T12:44:22.612486+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:22.612486+0000","last_peered":"2023-06-27T12:44:22.612486+0000","last_clean":"2023-06-27T12:44:22.612486+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:22.612486+0000","last_undegraded":"2023-06-27T12:44:22.612486+0000","last_fullsized":"2023-06-27T12:44:22.612486+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":351,"reported_epoch":368,"state":"active+undersized","last_fresh":"2023-06-27T12:44:22.613980+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:22.613980+0000","last_peered":"2023-06-27T12:44:22.613980+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:22.613980+0000","last_undegraded":"2023-06-27T12:44:22.613980+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"366'409","reported_seq":1269,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:22.621217+0000","last_change":"2023-06-27T12:44:22.621217+0000","last_active":"2023-06-27T12:44:22.621217+0000","last_peered":"2023-06-27T12:44:22.621217+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:22.621217+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7445765,"num_objects":52,"num_object_clones":46,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":320,"num_read_kb":198253,"num_write":288,"num_write_kb":84847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":52}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":599,"reported_epoch":368,"state":"active+clean","last_fresh":"2023-06-27T12:44:22.606031+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:22.606031+0000","last_peered":"2023-06-27T12:44:22.606031+0000","last_clean":"2023-06-27T12:44:22.606031+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:22.606031+0000","last_undegraded":"2023-06-27T12:44:22.606031+0000","last_fullsized":"2023-06-27T12:44:22.606031+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"354'257","reported_seq":655,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:22.593693+0000","last_change":"2023-06-27T12:44:22.614376+0000","last_active":"2023-06-27T12:44:22.593693+0000","last_peered":"2023-06-27T12:44:22.593693+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:22.593693+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9866138,"num_objects":19,"num_object_clones":19,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59443,"num_write":105,"num_write_kb":30476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":365,"reported_epoch":369,"state":"active+undersized","last_fresh":"2023-06-27T12:44:23.585974+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:23.585974+0000","last_peered":"2023-06-27T12:44:23.585974+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:23.585974+0000","last_undegraded":"2023-06-27T12:44:23.585974+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"366'219","reported_seq":1100,"reported_epoch":369,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:23.586297+0000","last_change":"2023-06-27T12:44:23.602870+0000","last_active":"2023-06-27T12:44:23.586297+0000","last_peered":"2023-06-27T12:44:23.586297+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:23.586297+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748456,"num_objects":29,"num_object_clones":25,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":116,"num_read_kb":123170,"num_write":146,"num_write_kb":42081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":29}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"361'144","reported_seq":611,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:22.595546+0000","last_change":"2023-06-27T12:44:22.708888+0000","last_active":"2023-06-27T12:44:22.595546+0000","last_peered":"2023-06-27T12:44:22.595546+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:22.595546+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3815321,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":101,"num_read_kb":56967,"num_write":58,"num_write_kb":16387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":622,"reported_epoch":369,"state":"active+undersized","last_fresh":"2023-06-27T12:44:23.578359+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:23.578359+0000","last_peered":"2023-06-27T12:44:23.578359+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:23.578359+0000","last_undegraded":"2023-06-27T12:44:23.578359+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":581,"reported_epoch":368,"state":"active+clean","last_fresh":"2023-06-27T12:44:22.593888+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:22.593888+0000","last_peered":"2023-06-27T12:44:22.593888+0000","last_clean":"2023-06-27T12:44:22.593888+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:22.593888+0000","last_undegraded":"2023-06-27T12:44:22.593888+0000","last_fullsized":"2023-06-27T12:44:22.593888+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"361'236","reported_seq":1093,"reported_epoch":369,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:23.582892+0000","last_change":"2023-06-27T12:44:23.589799+0000","last_active":"2023-06-27T12:44:23.582892+0000","last_peered":"2023-06-27T12:44:23.582892+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:23.582892+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5916328,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":122,"num_read_kb":132246,"num_write":125,"num_write_kb":37073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":115821525,"num_objects":376,"num_object_clones":329,"num_object_copies":1128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":2170,"num_read_kb":1587885,"num_write":1990,"num_write_kb":584473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3510,"ondisk_log_size":3510,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673350,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":295816,"kb_used_data":260800,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94076024,"statfs":{"total":96636764160,"available":96333848576,"internally_reserved":0,"allocated":267059200,"data_stored":309226456,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706055,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301948,"kb_used_data":266932,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94069892,"statfs":{"total":96636764160,"available":96327569408,"internally_reserved":0,"allocated":273338368,"data_stored":312081838,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673350,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":331828,"kb_used_data":296812,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94040012,"statfs":{"total":96636764160,"available":96296972288,"internally_reserved":0,"allocated":303935488,"data_stored":350652563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:28.384 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.e 3.c 3.8 3.3 3.2 2023-06-27T12:44:28.557 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:28.543+0000 7fe7dc422700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:28.560 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:28.556+0000 7fe7dc422700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:28.597 INFO:tasks.rados.rados.0.smithi138.stdout:1518: done (6 left) 2023-06-27T12:44:28.597 INFO:tasks.rados.rados.0.smithi138.stdout:1520: done (5 left) 2023-06-27T12:44:28.597 INFO:tasks.rados.rados.0.smithi138.stdout:1521: done (4 left) 2023-06-27T12:44:28.597 INFO:tasks.rados.rados.0.smithi138.stdout:1522: done (3 left) 2023-06-27T12:44:28.598 INFO:tasks.rados.rados.0.smithi138.stdout:1523: done (2 left) 2023-06-27T12:44:28.598 INFO:tasks.rados.rados.0.smithi138.stdout:1524: done (1 left) 2023-06-27T12:44:28.598 INFO:tasks.rados.rados.0.smithi138.stdout:1525: done (0 left) 2023-06-27T12:44:28.598 INFO:tasks.rados.rados.0.smithi138.stdout:1526: read oid 3 snap -1 2023-06-27T12:44:28.598 INFO:tasks.rados.rados.0.smithi138.stdout:1526: expect (ObjNum 707 snap 176 seq_num 707) 2023-06-27T12:44:28.598 INFO:tasks.rados.rados.0.smithi138.stdout:1527: write oid 37 current snap is 179 2023-06-27T12:44:28.598 INFO:tasks.rados.rados.0.smithi138.stdout:1527: seq_num 733 ranges {719531=727405,1893654=425877} 2023-06-27T12:44:28.608 INFO:tasks.rados.rados.0.smithi138.stdout:1527: writing smithi138138488-37 from 719531 to 1446936 tid 1 2023-06-27T12:44:28.615 INFO:tasks.rados.rados.0.smithi138.stdout:1527: writing smithi138138488-37 from 1893654 to 2319531 tid 2 2023-06-27T12:44:28.615 INFO:tasks.rados.rados.0.smithi138.stdout:1528: snap_remove snap 173 2023-06-27T12:44:28.771 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c,3.f] on osd.0 to force-backfill; instructing pg(s) [3.2,3.8] on osd.1 to force-backfill; instructing pg(s) [3.3,3.e] on osd.2 to force-backfill; 2023-06-27T12:44:28.771 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:44:29.599 INFO:tasks.rados.rados.0.smithi138.stdout:1528: done (2 left) 2023-06-27T12:44:29.599 INFO:tasks.rados.rados.0.smithi138.stdout:1529: read oid 45 snap -1 2023-06-27T12:44:29.600 INFO:tasks.rados.rados.0.smithi138.stdout:1529: expect (ObjNum 712 snap 177 seq_num 712) 2023-06-27T12:44:29.600 INFO:tasks.rados.rados.0.smithi138.stdout:1530: read oid 6 snap 175 2023-06-27T12:44:29.600 INFO:tasks.rados.rados.0.smithi138.stdout:1530: expect (ObjNum 667 snap 165 seq_num 667) 2023-06-27T12:44:29.600 INFO:tasks.rados.rados.0.smithi138.stdout:1531: snap_remove snap 176 2023-06-27T12:44:29.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:29.931 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:29.926+0000 7f3e82e1b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:29.934 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:29.930+0000 7f3e82e1b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:29.986 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:29.988 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:29.989 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:30.465 INFO:tasks.rados.rados.0.smithi138.stdout:1531: done (4 left) 2023-06-27T12:44:30.465 INFO:tasks.rados.rados.0.smithi138.stdout:1532: write (excl) oid 12 current snap is 179 2023-06-27T12:44:30.465 INFO:tasks.rados.rados.0.smithi138.stdout:1532: seq_num 734 ranges {736338=402654,1552136=539580} 2023-06-27T12:44:30.472 INFO:tasks.rados.rados.0.smithi138.stdout:1532: writing smithi138138488-12 from 736338 to 1138992 tid 1 2023-06-27T12:44:30.480 INFO:tasks.rados.rados.0.smithi138.stdout:1532: writing smithi138138488-12 from 1552136 to 2091716 tid 2 2023-06-27T12:44:30.480 INFO:tasks.rados.rados.0.smithi138.stdout:1533: write oid 31 current snap is 179 2023-06-27T12:44:30.481 INFO:tasks.rados.rados.0.smithi138.stdout:1533: seq_num 735 ranges {753145=477903,2010618=342527} 2023-06-27T12:44:30.489 INFO:tasks.rados.rados.0.smithi138.stdout:1533: writing smithi138138488-31 from 753145 to 1231048 tid 1 2023-06-27T12:44:30.497 INFO:tasks.rados.rados.0.smithi138.stdout:1533: writing smithi138138488-31 from 2010618 to 2353145 tid 2 2023-06-27T12:44:30.498 INFO:tasks.rados.rados.0.smithi138.stdout:1534: write (excl) oid 19 current snap is 179 2023-06-27T12:44:30.498 INFO:tasks.rados.rados.0.smithi138.stdout:1534: seq_num 736 ranges {769952=553152,2069100=300852} 2023-06-27T12:44:30.507 INFO:tasks.rados.rados.0.smithi138.stdout:1534: writing smithi138138488-19 from 769952 to 1323104 tid 1 2023-06-27T12:44:30.514 INFO:tasks.rados.rados.0.smithi138.stdout:1534: writing smithi138138488-19 from 2069100 to 2369952 tid 2 2023-06-27T12:44:30.515 INFO:tasks.rados.rados.0.smithi138.stdout:1535: snap_create 2023-06-27T12:44:30.515 INFO:tasks.rados.rados.0.smithi138.stdout:1526: expect (ObjNum 707 snap 176 seq_num 707) 2023-06-27T12:44:30.771 INFO:tasks.rados.rados.0.smithi138.stdout:1527: finishing write tid 1 to smithi138138488-37 2023-06-27T12:44:30.771 INFO:tasks.rados.rados.0.smithi138.stdout:1527: finishing write tid 2 to smithi138138488-37 2023-06-27T12:44:30.771 INFO:tasks.rados.rados.0.smithi138.stdout:1527: finishing write tid 3 to smithi138138488-37 2023-06-27T12:44:30.771 INFO:tasks.rados.rados.0.smithi138.stdout:1527: finishing write tid 4 to smithi138138488-37 2023-06-27T12:44:30.771 INFO:tasks.rados.rados.0.smithi138.stdout:1527: oid 37 updating version 0 to 309 2023-06-27T12:44:30.772 INFO:tasks.rados.rados.0.smithi138.stdout:1527: oid 37 version 309 is already newer than 309 2023-06-27T12:44:30.772 INFO:tasks.rados.rados.0.smithi138.stdout:1527: oid 37 version 309 is already newer than 309 2023-06-27T12:44:30.772 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 309 (ObjNum 733 snap 179 seq_num 733) dirty exists 2023-06-27T12:44:30.772 INFO:tasks.rados.rados.0.smithi138.stdout:1527: left oid 37 (ObjNum 733 snap 179 seq_num 733) 2023-06-27T12:44:30.772 INFO:tasks.rados.rados.0.smithi138.stdout:1530: expect (ObjNum 667 snap 165 seq_num 667) 2023-06-27T12:44:30.932 INFO:tasks.rados.rados.0.smithi138.stdout:1529: expect (ObjNum 712 snap 177 seq_num 712) 2023-06-27T12:44:31.079 INFO:tasks.rados.rados.0.smithi138.stdout:1534: finishing write tid 1 to smithi138138488-19 2023-06-27T12:44:31.080 INFO:tasks.rados.rados.0.smithi138.stdout:1532: finishing write tid 1 to smithi138138488-12 2023-06-27T12:44:31.080 INFO:tasks.rados.rados.0.smithi138.stdout:1533: finishing write tid 1 to smithi138138488-31 2023-06-27T12:44:31.080 INFO:tasks.rados.rados.0.smithi138.stdout:1533: finishing write tid 2 to smithi138138488-31 2023-06-27T12:44:31.080 INFO:tasks.rados.rados.0.smithi138.stdout:1533: finishing write tid 3 to smithi138138488-31 2023-06-27T12:44:31.080 INFO:tasks.rados.rados.0.smithi138.stdout:1533: finishing write tid 4 to smithi138138488-31 2023-06-27T12:44:31.081 INFO:tasks.rados.rados.0.smithi138.stdout:1533: oid 31 updating version 0 to 455 2023-06-27T12:44:31.081 INFO:tasks.rados.rados.0.smithi138.stdout:1533: oid 31 version 455 is already newer than 455 2023-06-27T12:44:31.081 INFO:tasks.rados.rados.0.smithi138.stdout:1533: oid 31 version 455 is already newer than 455 2023-06-27T12:44:31.081 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 455 (ObjNum 735 snap 179 seq_num 735) dirty exists 2023-06-27T12:44:31.081 INFO:tasks.rados.rados.0.smithi138.stdout:1533: left oid 31 (ObjNum 735 snap 179 seq_num 735) 2023-06-27T12:44:31.081 INFO:tasks.rados.rados.0.smithi138.stdout:1534: finishing write tid 2 to smithi138138488-19 2023-06-27T12:44:31.081 INFO:tasks.rados.rados.0.smithi138.stdout:1534: finishing write tid 3 to smithi138138488-19 2023-06-27T12:44:31.082 INFO:tasks.rados.rados.0.smithi138.stdout:1534: finishing write tid 4 to smithi138138488-19 2023-06-27T12:44:31.082 INFO:tasks.rados.rados.0.smithi138.stdout:1534: oid 19 updating version 0 to 438 2023-06-27T12:44:31.082 INFO:tasks.rados.rados.0.smithi138.stdout:1534: oid 19 version 438 is already newer than 438 2023-06-27T12:44:31.082 INFO:tasks.rados.rados.0.smithi138.stdout:1534: oid 19 version 438 is already newer than 436 2023-06-27T12:44:31.082 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 438 (ObjNum 736 snap 179 seq_num 736) dirty exists 2023-06-27T12:44:31.082 INFO:tasks.rados.rados.0.smithi138.stdout:1534: left oid 19 (ObjNum 736 snap 179 seq_num 736) 2023-06-27T12:44:31.082 INFO:tasks.rados.rados.0.smithi138.stdout:1532: finishing write tid 2 to smithi138138488-12 2023-06-27T12:44:31.083 INFO:tasks.rados.rados.0.smithi138.stdout:1532: finishing write tid 3 to smithi138138488-12 2023-06-27T12:44:31.083 INFO:tasks.rados.rados.0.smithi138.stdout:1532: finishing write tid 4 to smithi138138488-12 2023-06-27T12:44:31.083 INFO:tasks.rados.rados.0.smithi138.stdout:1532: oid 12 updating version 0 to 315 2023-06-27T12:44:31.083 INFO:tasks.rados.rados.0.smithi138.stdout:1532: oid 12 version 315 is already newer than 315 2023-06-27T12:44:31.083 INFO:tasks.rados.rados.0.smithi138.stdout:1532: oid 12 version 315 is already newer than 312 2023-06-27T12:44:31.083 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 315 (ObjNum 734 snap 179 seq_num 734) dirty exists 2023-06-27T12:44:31.084 INFO:tasks.rados.rados.0.smithi138.stdout:1532: left oid 12 (ObjNum 734 snap 179 seq_num 734) 2023-06-27T12:44:31.483 INFO:tasks.rados.rados.0.smithi138.stdout:1526: done (7 left) 2023-06-27T12:44:31.484 INFO:tasks.rados.rados.0.smithi138.stdout:1527: done (6 left) 2023-06-27T12:44:31.484 INFO:tasks.rados.rados.0.smithi138.stdout:1529: done (5 left) 2023-06-27T12:44:31.484 INFO:tasks.rados.rados.0.smithi138.stdout:1530: done (4 left) 2023-06-27T12:44:31.484 INFO:tasks.rados.rados.0.smithi138.stdout:1532: done (3 left) 2023-06-27T12:44:31.484 INFO:tasks.rados.rados.0.smithi138.stdout:1533: done (2 left) 2023-06-27T12:44:31.485 INFO:tasks.rados.rados.0.smithi138.stdout:1534: done (1 left) 2023-06-27T12:44:31.485 INFO:tasks.rados.rados.0.smithi138.stdout:1535: done (0 left) 2023-06-27T12:44:31.485 INFO:tasks.rados.rados.0.smithi138.stdout:1536: write oid 14 current snap is 180 2023-06-27T12:44:31.485 INFO:tasks.rados.rados.0.smithi138.stdout:1536: seq_num 737 ranges {786759=628401,2011229=375530} 2023-06-27T12:44:31.494 INFO:tasks.rados.rados.0.smithi138.stdout:1536: writing smithi138138488-14 from 786759 to 1415160 tid 1 2023-06-27T12:44:31.501 INFO:tasks.rados.rados.0.smithi138.stdout:1536: writing smithi138138488-14 from 2011229 to 2386759 tid 2 2023-06-27T12:44:31.501 INFO:tasks.rados.rados.0.smithi138.stdout:1537: snap_create 2023-06-27T12:44:31.585 INFO:tasks.rados.rados.0.smithi138.stdout:1536: finishing write tid 1 to smithi138138488-14 2023-06-27T12:44:31.585 INFO:tasks.rados.rados.0.smithi138.stdout:1536: finishing write tid 2 to smithi138138488-14 2023-06-27T12:44:31.585 INFO:tasks.rados.rados.0.smithi138.stdout:1536: finishing write tid 3 to smithi138138488-14 2023-06-27T12:44:31.586 INFO:tasks.rados.rados.0.smithi138.stdout:1536: finishing write tid 4 to smithi138138488-14 2023-06-27T12:44:31.586 INFO:tasks.rados.rados.0.smithi138.stdout:1536: oid 14 updating version 0 to 442 2023-06-27T12:44:31.587 INFO:tasks.rados.rados.0.smithi138.stdout:1536: oid 14 version 442 is already newer than 442 2023-06-27T12:44:31.587 INFO:tasks.rados.rados.0.smithi138.stdout:1536: oid 14 version 442 is already newer than 442 2023-06-27T12:44:31.587 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 442 (ObjNum 737 snap 180 seq_num 737) dirty exists 2023-06-27T12:44:31.587 INFO:tasks.rados.rados.0.smithi138.stdout:1536: left oid 14 (ObjNum 737 snap 180 seq_num 737) 2023-06-27T12:44:31.990 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:31.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~237s 2023-06-27T12:44:32.120 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:32.115+0000 7fe6f3eae700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:32.123 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:32.119+0000 7fe6f3eae700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:32.168 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:32.170 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:32.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:32.484 INFO:tasks.rados.rados.0.smithi138.stdout:1536: done (1 left) 2023-06-27T12:44:32.484 INFO:tasks.rados.rados.0.smithi138.stdout:1537: done (0 left) 2023-06-27T12:44:32.484 INFO:tasks.rados.rados.0.smithi138.stdout:1538: write oid 42 current snap is 181 2023-06-27T12:44:32.484 INFO:tasks.rados.rados.0.smithi138.stdout:1538: seq_num 738 ranges {403566=420003,1386064=546918} 2023-06-27T12:44:32.490 INFO:tasks.rados.rados.0.smithi138.stdout:1538: writing smithi138138488-42 from 403566 to 823569 tid 1 2023-06-27T12:44:32.499 INFO:tasks.rados.rados.0.smithi138.stdout:1538: writing smithi138138488-42 from 1386064 to 1932982 tid 2 2023-06-27T12:44:32.499 INFO:tasks.rados.rados.0.smithi138.stdout:1539: snap_remove snap 178 2023-06-27T12:44:33.484 INFO:tasks.rados.rados.0.smithi138.stdout:1539: done (1 left) 2023-06-27T12:44:33.485 INFO:tasks.rados.rados.0.smithi138.stdout:1540: read oid 39 snap -1 2023-06-27T12:44:33.485 INFO:tasks.rados.rados.0.smithi138.stdout:1540: expect (ObjNum 635 snap 153 seq_num 635) 2023-06-27T12:44:33.486 INFO:tasks.rados.rados.0.smithi138.stdout:1541: write oid 25 current snap is 181 2023-06-27T12:44:33.486 INFO:tasks.rados.rados.0.smithi138.stdout:1538: finishing write tid 1 to smithi138138488-42 2023-06-27T12:44:33.486 INFO:tasks.rados.rados.0.smithi138.stdout:1541: seq_num 739 ranges {420373=495252,1444546=690576} 2023-06-27T12:44:33.491 INFO:tasks.rados.rados.0.smithi138.stdout:1541: writing smithi138138488-25 from 420373 to 915625 tid 1 2023-06-27T12:44:33.501 INFO:tasks.rados.rados.0.smithi138.stdout:1541: writing smithi138138488-25 from 1444546 to 2135122 tid 2 2023-06-27T12:44:33.501 INFO:tasks.rados.rados.0.smithi138.stdout:1542: read oid 43 snap -1 2023-06-27T12:44:33.501 INFO:tasks.rados.rados.0.smithi138.stdout:1542: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:44:33.501 INFO:tasks.rados.rados.0.smithi138.stdout:1543: snap_create 2023-06-27T12:44:33.502 INFO:tasks.rados.rados.0.smithi138.stdout:1538: finishing write tid 2 to smithi138138488-42 2023-06-27T12:44:33.502 INFO:tasks.rados.rados.0.smithi138.stdout:1538: finishing write tid 3 to smithi138138488-42 2023-06-27T12:44:33.502 INFO:tasks.rados.rados.0.smithi138.stdout:1538: finishing write tid 4 to smithi138138488-42 2023-06-27T12:44:33.502 INFO:tasks.rados.rados.0.smithi138.stdout:1538: oid 42 updating version 0 to 213 2023-06-27T12:44:33.502 INFO:tasks.rados.rados.0.smithi138.stdout:1538: oid 42 version 213 is already newer than 213 2023-06-27T12:44:33.502 INFO:tasks.rados.rados.0.smithi138.stdout:1538: oid 42 version 213 is already newer than 213 2023-06-27T12:44:33.503 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 213 (ObjNum 738 snap 181 seq_num 738) dirty exists 2023-06-27T12:44:33.503 INFO:tasks.rados.rados.0.smithi138.stdout:1538: left oid 42 (ObjNum 738 snap 181 seq_num 738) 2023-06-27T12:44:33.559 INFO:tasks.rados.rados.0.smithi138.stdout:1542: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:44:33.728 INFO:tasks.rados.rados.0.smithi138.stdout:1540: expect (ObjNum 635 snap 153 seq_num 635) 2023-06-27T12:44:33.783 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:33.783 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:33.784 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:33.867 INFO:tasks.rados.rados.0.smithi138.stdout:1541: finishing write tid 1 to smithi138138488-25 2023-06-27T12:44:33.867 INFO:tasks.rados.rados.0.smithi138.stdout:1541: finishing write tid 2 to smithi138138488-25 2023-06-27T12:44:33.867 INFO:tasks.rados.rados.0.smithi138.stdout:1541: finishing write tid 3 to smithi138138488-25 2023-06-27T12:44:33.867 INFO:tasks.rados.rados.0.smithi138.stdout:1541: finishing write tid 4 to smithi138138488-25 2023-06-27T12:44:33.868 INFO:tasks.rados.rados.0.smithi138.stdout:1541: oid 25 updating version 0 to 153 2023-06-27T12:44:33.868 INFO:tasks.rados.rados.0.smithi138.stdout:1541: oid 25 version 153 is already newer than 153 2023-06-27T12:44:33.868 INFO:tasks.rados.rados.0.smithi138.stdout:1541: oid 25 version 153 is already newer than 153 2023-06-27T12:44:33.868 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 153 (ObjNum 739 snap 181 seq_num 739) dirty exists 2023-06-27T12:44:33.868 INFO:tasks.rados.rados.0.smithi138.stdout:1541: left oid 25 (ObjNum 739 snap 181 seq_num 739) 2023-06-27T12:44:33.913 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:33.906+0000 7faf91f90700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:33.916 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:33.913+0000 7faf91f90700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:34.113 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:34.115 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:34.127 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":572,"stamp":"2023-06-27T12:44:33.998673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":109721409,"num_objects":385,"num_object_clones":338,"num_object_copies":1155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":2235,"num_read_kb":1652180,"num_write":2046,"num_write_kb":597835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3727,"ondisk_log_size":3727,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1141600,"kb_used_data":1003576,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376345760,"statfs":{"total":386547056640,"available":385378058240,"internally_reserved":0,"allocated":1027661824,"data_stored":1182514389,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000962"},"pg_stats":[{"pgid":"3.f","version":"361'54","reported_seq":887,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.620926+0000","last_change":"2023-06-27T12:44:27.661263+0000","last_active":"2023-06-27T12:44:28.620926+0000","last_peered":"2023-06-27T12:44:28.620926+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:28.620926+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3731286,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":13,"num_read_kb":7,"num_write":33,"num_write_kb":9288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"372'249","reported_seq":1187,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.613279+0000","last_change":"2023-06-27T12:44:27.717451+0000","last_active":"2023-06-27T12:44:28.613279+0000","last_peered":"2023-06-27T12:44:28.613279+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:28.613279+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13681459,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":165,"num_read_kb":204993,"num_write":176,"num_write_kb":53586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":32}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"374'309","reported_seq":1160,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.661072+0000","last_change":"2023-06-27T12:44:27.674985+0000","last_active":"2023-06-27T12:44:28.661072+0000","last_peered":"2023-06-27T12:44:28.661072+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:28.661072+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10336866,"num_objects":34,"num_object_clones":29,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":256,"num_read_kb":171535,"num_write":192,"num_write_kb":55994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"371'204","reported_seq":1103,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.635357+0000","last_change":"2023-06-27T12:44:28.724628+0000","last_active":"2023-06-27T12:44:28.635357+0000","last_peered":"2023-06-27T12:44:28.635357+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:28.635357+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6504573,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":196,"num_read_kb":141766,"num_write":135,"num_write_kb":38638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":30}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":631,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.638993+0000","last_change":"2023-06-27T12:44:27.648415+0000","last_active":"2023-06-27T12:44:28.638993+0000","last_peered":"2023-06-27T12:44:28.638993+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:28.638993+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"373'445","reported_seq":1189,"reported_epoch":374,"state":"active+clean","last_fresh":"2023-06-27T12:44:28.626338+0000","last_change":"2023-06-27T12:44:28.642882+0000","last_active":"2023-06-27T12:44:28.626338+0000","last_peered":"2023-06-27T12:44:28.626338+0000","last_clean":"2023-06-27T12:44:28.626338+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:28.626338+0000","last_undegraded":"2023-06-27T12:44:28.626338+0000","last_fullsized":"2023-06-27T12:44:28.626338+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12101634,"num_objects":48,"num_object_clones":42,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":332,"num_read_kb":211119,"num_write":261,"num_write_kb":75029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"371'220","reported_seq":992,"reported_epoch":374,"state":"active+clean","last_fresh":"2023-06-27T12:44:28.640942+0000","last_change":"2023-06-27T12:44:28.729396+0000","last_active":"2023-06-27T12:44:28.640942+0000","last_peered":"2023-06-27T12:44:28.640942+0000","last_clean":"2023-06-27T12:44:28.640942+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:28.640942+0000","last_undegraded":"2023-06-27T12:44:28.640942+0000","last_fullsized":"2023-06-27T12:44:28.640942+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6387155,"num_objects":27,"num_object_clones":24,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":85553,"num_write":139,"num_write_kb":40042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":923,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.631686+0000","last_change":"2023-06-27T12:44:27.637254+0000","last_active":"2023-06-27T12:44:28.631686+0000","last_peered":"2023-06-27T12:44:28.631686+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:28.631686+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":632,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-06-27T12:44:28.611591+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:28.611591+0000","last_peered":"2023-06-27T12:44:28.611591+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:28.611591+0000","last_undegraded":"2023-06-27T12:44:28.611591+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":831,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-06-27T12:44:28.618096+0000","last_change":"2023-06-27T12:44:27.677644+0000","last_active":"2023-06-27T12:44:28.618096+0000","last_peered":"2023-06-27T12:44:28.618096+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:28.618096+0000","last_undegraded":"2023-06-27T12:44:28.618096+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":607,"reported_epoch":374,"state":"active+clean","last_fresh":"2023-06-27T12:44:28.625010+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:28.625010+0000","last_peered":"2023-06-27T12:44:28.625010+0000","last_clean":"2023-06-27T12:44:28.625010+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:28.625010+0000","last_undegraded":"2023-06-27T12:44:28.625010+0000","last_fullsized":"2023-06-27T12:44:28.625010+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"370'201","reported_seq":1067,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.608200+0000","last_change":"2023-06-27T12:44:27.618969+0000","last_active":"2023-06-27T12:44:28.608200+0000","last_peered":"2023-06-27T12:44:28.608200+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:28.608200+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8135017,"num_objects":22,"num_object_clones":17,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":82,"num_read_kb":70439,"num_write":128,"num_write_kb":35298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":22}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":594,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-06-27T12:44:28.631360+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:28.631360+0000","last_peered":"2023-06-27T12:44:28.631360+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:28.631360+0000","last_undegraded":"2023-06-27T12:44:28.631360+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"372'459","reported_seq":754,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.616519+0000","last_change":"2023-06-27T12:44:27.720917+0000","last_active":"2023-06-27T12:44:28.616519+0000","last_peered":"2023-06-27T12:44:28.616519+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:28.616519+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2890936,"num_objects":28,"num_object_clones":26,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":75,"num_read_kb":49591,"num_write":136,"num_write_kb":42509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":591,"reported_epoch":374,"state":"active+clean","last_fresh":"2023-06-27T12:44:28.627382+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:28.627382+0000","last_peered":"2023-06-27T12:44:28.627382+0000","last_clean":"2023-06-27T12:44:28.627382+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:28.627382+0000","last_undegraded":"2023-06-27T12:44:28.627382+0000","last_fullsized":"2023-06-27T12:44:28.627382+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":359,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-06-27T12:44:28.620976+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:28.620976+0000","last_peered":"2023-06-27T12:44:28.620976+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:28.620976+0000","last_undegraded":"2023-06-27T12:44:28.620976+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"372'423","reported_seq":1289,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.623006+0000","last_change":"2023-06-27T12:44:27.741799+0000","last_active":"2023-06-27T12:44:28.623006+0000","last_peered":"2023-06-27T12:44:28.623006+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:28.623006+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6151659,"num_objects":53,"num_object_clones":47,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":322,"num_read_kb":198254,"num_write":292,"num_write_kb":85973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":53}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":607,"reported_epoch":374,"state":"active+clean","last_fresh":"2023-06-27T12:44:28.638111+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:28.638111+0000","last_peered":"2023-06-27T12:44:28.638111+0000","last_clean":"2023-06-27T12:44:28.638111+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:28.638111+0000","last_undegraded":"2023-06-27T12:44:28.638111+0000","last_fullsized":"2023-06-27T12:44:28.638111+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"373'271","reported_seq":685,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.623400+0000","last_change":"2023-06-27T12:44:27.895953+0000","last_active":"2023-06-27T12:44:28.623400+0000","last_peered":"2023-06-27T12:44:28.623400+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:28.623400+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8925012,"num_objects":20,"num_object_clones":20,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":67306,"num_write":117,"num_write_kb":33414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":375,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-06-27T12:44:28.615339+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:28.615339+0000","last_peered":"2023-06-27T12:44:28.615339+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:28.615339+0000","last_undegraded":"2023-06-27T12:44:28.615339+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"373'232","reported_seq":1122,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.627703+0000","last_change":"2023-06-27T12:44:27.833238+0000","last_active":"2023-06-27T12:44:28.627703+0000","last_peered":"2023-06-27T12:44:28.627703+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:28.627703+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12051180,"num_objects":29,"num_object_clones":25,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":134749,"num_write":150,"num_write_kb":43061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":29}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"361'144","reported_seq":626,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.618836+0000","last_change":"2023-06-27T12:44:27.647714+0000","last_active":"2023-06-27T12:44:28.618836+0000","last_peered":"2023-06-27T12:44:28.618836+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:28.618836+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3815321,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":101,"num_read_kb":56967,"num_write":58,"num_write_kb":16387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":632,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-06-27T12:44:28.613507+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:28.613507+0000","last_peered":"2023-06-27T12:44:28.613507+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:28.613507+0000","last_undegraded":"2023-06-27T12:44:28.613507+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":591,"reported_epoch":374,"state":"active+clean","last_fresh":"2023-06-27T12:44:28.636694+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:28.636694+0000","last_peered":"2023-06-27T12:44:28.636694+0000","last_clean":"2023-06-27T12:44:28.636694+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:28.636694+0000","last_undegraded":"2023-06-27T12:44:28.636694+0000","last_fullsized":"2023-06-27T12:44:28.636694+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"361'236","reported_seq":1108,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:28.612375+0000","last_change":"2023-06-27T12:44:27.608203+0000","last_active":"2023-06-27T12:44:28.612375+0000","last_peered":"2023-06-27T12:44:28.612375+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:28.612375+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5916328,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":124,"num_read_kb":135931,"num_write":125,"num_write_kb":37073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":109721409,"num_objects":385,"num_object_clones":338,"num_object_copies":1155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":19,"num_read":2235,"num_read_kb":1652180,"num_write":2046,"num_write_kb":597835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3630,"ondisk_log_size":3630,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673355,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303152,"kb_used_data":268136,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94068688,"statfs":{"total":96636764160,"available":96326336512,"internally_reserved":0,"allocated":274571264,"data_stored":318145013,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":313632,"kb_used_data":277992,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94058208,"statfs":{"total":96636764160,"available":96315604992,"internally_reserved":0,"allocated":284663808,"data_stored":324466965,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673355,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":343536,"kb_used_data":307088,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94028304,"statfs":{"total":96636764160,"available":96284983296,"internally_reserved":0,"allocated":314458112,"data_stored":362862859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:34.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.e 3.c 3.b 3.0 3.2 3.1 3.4 3.6 2023-06-27T12:44:34.172 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:34.269 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:34.252+0000 7f8b8b3a2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:34.272 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:34.269+0000 7f8b8b3a2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:34.312 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:34.307+0000 7fcff4002700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:34.315 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:34.311+0000 7fcff2da0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:34.362 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:34.365 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:34.366 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:34.486 INFO:tasks.rados.rados.0.smithi138.stdout:1538: done (4 left) 2023-06-27T12:44:34.486 INFO:tasks.rados.rados.0.smithi138.stdout:1540: done (3 left) 2023-06-27T12:44:34.486 INFO:tasks.rados.rados.0.smithi138.stdout:1541: done (2 left) 2023-06-27T12:44:34.487 INFO:tasks.rados.rados.0.smithi138.stdout:1542: done (1 left) 2023-06-27T12:44:34.487 INFO:tasks.rados.rados.0.smithi138.stdout:1543: done (0 left) 2023-06-27T12:44:34.487 INFO:tasks.rados.rados.0.smithi138.stdout:1544: delete oid 13 current snap is 182 2023-06-27T12:44:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.c] on osd.0 to force-recovery; instructing pg(s) [3.1,3.2] on osd.1 to force-recovery; instructing pg(s) [3.0,3.4,3.6,3.e] on osd.2 to force-recovery; 2023-06-27T12:44:34.488 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:44:34.514 INFO:tasks.rados.rados.0.smithi138.stdout:1544: done (0 left) 2023-06-27T12:44:34.514 INFO:tasks.rados.rados.0.smithi138.stdout:1545: write (excl) oid 10 current snap is 182 2023-06-27T12:44:34.514 INFO:tasks.rados.rados.0.smithi138.stdout:1545: seq_num 741 ranges {453987=645750,1845157=608830} 2023-06-27T12:44:34.520 INFO:tasks.rados.rados.0.smithi138.stdout:1545: writing smithi138138488-10 from 453987 to 1099737 tid 1 2023-06-27T12:44:34.530 INFO:tasks.rados.rados.0.smithi138.stdout:1545: writing smithi138138488-10 from 1845157 to 2453987 tid 2 2023-06-27T12:44:34.531 INFO:tasks.rados.rados.0.smithi138.stdout:1546: write (excl) oid 1 current snap is 182 2023-06-27T12:44:34.531 INFO:tasks.rados.rados.0.smithi138.stdout:1546: seq_num 742 ranges {470794=720999,1903639=554256} 2023-06-27T12:44:34.540 INFO:tasks.rados.rados.0.smithi138.stdout:1546: writing smithi138138488-1 from 470794 to 1191793 tid 1 2023-06-27T12:44:34.549 INFO:tasks.rados.rados.0.smithi138.stdout:1546: writing smithi138138488-1 from 1903639 to 2457895 tid 2 2023-06-27T12:44:34.550 INFO:tasks.rados.rados.0.smithi138.stdout:1547: read oid 29 snap -1 2023-06-27T12:44:34.550 INFO:tasks.rados.rados.0.smithi138.stdout:1547: expect (ObjNum 703 snap 171 seq_num 703) 2023-06-27T12:44:34.550 INFO:tasks.rados.rados.0.smithi138.stdout:1548: snap_create 2023-06-27T12:44:34.568 INFO:tasks.rados.rados.0.smithi138.stdout:1547: expect (ObjNum 703 snap 171 seq_num 703) 2023-06-27T12:44:34.790 INFO:tasks.rados.rados.0.smithi138.stdout:1546: finishing write tid 1 to smithi138138488-1 2023-06-27T12:44:34.790 INFO:tasks.rados.rados.0.smithi138.stdout:1545: finishing write tid 1 to smithi138138488-10 2023-06-27T12:44:34.791 INFO:tasks.rados.rados.0.smithi138.stdout:1546: finishing write tid 2 to smithi138138488-1 2023-06-27T12:44:34.791 INFO:tasks.rados.rados.0.smithi138.stdout:1546: finishing write tid 3 to smithi138138488-1 2023-06-27T12:44:34.791 INFO:tasks.rados.rados.0.smithi138.stdout:1546: finishing write tid 4 to smithi138138488-1 2023-06-27T12:44:34.791 INFO:tasks.rados.rados.0.smithi138.stdout:1546: oid 1 updating version 0 to 445 2023-06-27T12:44:34.791 INFO:tasks.rados.rados.0.smithi138.stdout:1546: oid 1 updating version 445 to 447 2023-06-27T12:44:34.791 INFO:tasks.rados.rados.0.smithi138.stdout:1546: oid 1 version 447 is already newer than 447 2023-06-27T12:44:34.791 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 447 (ObjNum 742 snap 182 seq_num 742) dirty exists 2023-06-27T12:44:34.791 INFO:tasks.rados.rados.0.smithi138.stdout:1546: left oid 1 (ObjNum 742 snap 182 seq_num 742) 2023-06-27T12:44:34.792 INFO:tasks.rados.rados.0.smithi138.stdout:1545: finishing write tid 2 to smithi138138488-10 2023-06-27T12:44:34.792 INFO:tasks.rados.rados.0.smithi138.stdout:1545: finishing write tid 3 to smithi138138488-10 2023-06-27T12:44:34.792 INFO:tasks.rados.rados.0.smithi138.stdout:1545: finishing write tid 4 to smithi138138488-10 2023-06-27T12:44:34.792 INFO:tasks.rados.rados.0.smithi138.stdout:1545: oid 10 updating version 0 to 233 2023-06-27T12:44:34.792 INFO:tasks.rados.rados.0.smithi138.stdout:1545: oid 10 updating version 233 to 236 2023-06-27T12:44:34.792 INFO:tasks.rados.rados.0.smithi138.stdout:1545: oid 10 version 236 is already newer than 236 2023-06-27T12:44:34.792 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 236 (ObjNum 741 snap 182 seq_num 741) dirty exists 2023-06-27T12:44:34.793 INFO:tasks.rados.rados.0.smithi138.stdout:1545: left oid 10 (ObjNum 741 snap 182 seq_num 741) 2023-06-27T12:44:35.486 INFO:tasks.rados.rados.0.smithi138.stdout:1545: done (3 left) 2023-06-27T12:44:35.486 INFO:tasks.rados.rados.0.smithi138.stdout:1546: done (2 left) 2023-06-27T12:44:35.486 INFO:tasks.rados.rados.0.smithi138.stdout:1547: done (1 left) 2023-06-27T12:44:35.487 INFO:tasks.rados.rados.0.smithi138.stdout:1548: done (0 left) 2023-06-27T12:44:35.487 INFO:tasks.rados.rados.0.smithi138.stdout:1549: snap_create 2023-06-27T12:44:36.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:36.489 INFO:tasks.rados.rados.0.smithi138.stdout:1549: done (0 left) 2023-06-27T12:44:36.489 INFO:tasks.rados.rados.0.smithi138.stdout:1550: delete oid 29 current snap is 184 2023-06-27T12:44:36.513 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:36.506+0000 7f3c41a5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:36.516 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:36.513+0000 7f3c41a5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:36.599 INFO:tasks.rados.rados.0.smithi138.stdout:1550: done (0 left) 2023-06-27T12:44:36.599 INFO:tasks.rados.rados.0.smithi138.stdout:1551: read oid 43 snap -1 2023-06-27T12:44:36.600 INFO:tasks.rados.rados.0.smithi138.stdout:1551: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:44:36.600 INFO:tasks.rados.rados.0.smithi138.stdout:1552: write oid 3 current snap is 184 2023-06-27T12:44:36.600 INFO:tasks.rados.rados.0.smithi138.stdout:1552: seq_num 744 ranges {504408=471497,1620603=557925} 2023-06-27T12:44:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:36.607 INFO:tasks.rados.rados.0.smithi138.stdout:1552: writing smithi138138488-3 from 504408 to 975905 tid 1 2023-06-27T12:44:36.608 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:36.609 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:36.614 INFO:tasks.rados.rados.0.smithi138.stdout:1552: writing smithi138138488-3 from 1620603 to 2178528 tid 2 2023-06-27T12:44:36.615 INFO:tasks.rados.rados.0.smithi138.stdout:1553: read oid 18 snap -1 2023-06-27T12:44:36.615 INFO:tasks.rados.rados.0.smithi138.stdout:1553: expect deleted 2023-06-27T12:44:36.615 INFO:tasks.rados.rados.0.smithi138.stdout:1554: read oid 29 snap -1 2023-06-27T12:44:36.615 INFO:tasks.rados.rados.0.smithi138.stdout:1554: expect deleted 2023-06-27T12:44:36.615 INFO:tasks.rados.rados.0.smithi138.stdout:1555: delete oid 28 current snap is 184 2023-06-27T12:44:36.618 INFO:tasks.rados.rados.0.smithi138.stdout:1551: expect (ObjNum 616 snap 146 seq_num 616) 2023-06-27T12:44:36.784 INFO:tasks.rados.rados.0.smithi138.stdout:1552: finishing write tid 1 to smithi138138488-3 2023-06-27T12:44:36.784 INFO:tasks.rados.rados.0.smithi138.stdout:1552: finishing write tid 2 to smithi138138488-3 2023-06-27T12:44:36.784 INFO:tasks.rados.rados.0.smithi138.stdout:1552: finishing write tid 3 to smithi138138488-3 2023-06-27T12:44:36.785 INFO:tasks.rados.rados.0.smithi138.stdout:1552: finishing write tid 4 to smithi138138488-3 2023-06-27T12:44:36.785 INFO:tasks.rados.rados.0.smithi138.stdout:1552: oid 3 updating version 0 to 238 2023-06-27T12:44:36.785 INFO:tasks.rados.rados.0.smithi138.stdout:1552: oid 3 version 238 is already newer than 238 2023-06-27T12:44:36.785 INFO:tasks.rados.rados.0.smithi138.stdout:1552: oid 3 version 238 is already newer than 238 2023-06-27T12:44:36.785 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 238 (ObjNum 744 snap 184 seq_num 744) dirty exists 2023-06-27T12:44:36.785 INFO:tasks.rados.rados.0.smithi138.stdout:1552: left oid 3 (ObjNum 744 snap 184 seq_num 744) 2023-06-27T12:44:36.785 INFO:tasks.rados.rados.0.smithi138.stdout:1551: done (4 left) 2023-06-27T12:44:36.786 INFO:tasks.rados.rados.0.smithi138.stdout:1552: done (3 left) 2023-06-27T12:44:36.786 INFO:tasks.rados.rados.0.smithi138.stdout:1553: done (2 left) 2023-06-27T12:44:36.786 INFO:tasks.rados.rados.0.smithi138.stdout:1554: done (1 left) 2023-06-27T12:44:36.786 INFO:tasks.rados.rados.0.smithi138.stdout:1555: done (0 left) 2023-06-27T12:44:36.786 INFO:tasks.rados.rados.0.smithi138.stdout:1556: snap_remove snap 172 2023-06-27T12:44:37.494 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~242s 2023-06-27T12:44:37.622 INFO:tasks.rados.rados.0.smithi138.stdout:1556: done (0 left) 2023-06-27T12:44:37.622 INFO:tasks.rados.rados.0.smithi138.stdout:1557: snap_remove snap 180 2023-06-27T12:44:38.610 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:38.633 INFO:tasks.rados.rados.0.smithi138.stdout:1557: done (0 left) 2023-06-27T12:44:38.633 INFO:tasks.rados.rados.0.smithi138.stdout:1558: delete oid 6 current snap is 184 2023-06-27T12:44:38.677 INFO:tasks.rados.rados.0.smithi138.stdout:1558: done (0 left) 2023-06-27T12:44:38.677 INFO:tasks.rados.rados.0.smithi138.stdout:1559: write oid 47 current snap is 184 2023-06-27T12:44:38.677 INFO:tasks.rados.rados.0.smithi138.stdout:1559: seq_num 747 ranges {554829=413597,1396049=705252} 2023-06-27T12:44:38.682 INFO:tasks.rados.rados.0.smithi138.stdout:1559: writing smithi138138488-47 from 554829 to 968426 tid 1 2023-06-27T12:44:38.688 INFO:tasks.rados.rados.0.smithi138.stdout:1559: writing smithi138138488-47 from 1396049 to 2101301 tid 2 2023-06-27T12:44:38.689 INFO:tasks.rados.rados.0.smithi138.stdout:1560: read oid 20 snap -1 2023-06-27T12:44:38.689 INFO:tasks.rados.rados.0.smithi138.stdout:1560: expect (ObjNum 726 snap 178 seq_num 726) 2023-06-27T12:44:38.689 INFO:tasks.rados.rados.0.smithi138.stdout:1561: snap_create 2023-06-27T12:44:38.718 INFO:tasks.rados.rados.0.smithi138.stdout:1559: finishing write tid 1 to smithi138138488-47 2023-06-27T12:44:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:38.757+0000 7fab942b0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:38.760+0000 7fab942b0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:38.766 INFO:tasks.rados.rados.0.smithi138.stdout:1559: finishing write tid 2 to smithi138138488-47 2023-06-27T12:44:38.767 INFO:tasks.rados.rados.0.smithi138.stdout:1559: finishing write tid 3 to smithi138138488-47 2023-06-27T12:44:38.768 INFO:tasks.rados.rados.0.smithi138.stdout:1559: finishing write tid 4 to smithi138138488-47 2023-06-27T12:44:38.769 INFO:tasks.rados.rados.0.smithi138.stdout:1559: oid 47 updating version 0 to 261 2023-06-27T12:44:38.769 INFO:tasks.rados.rados.0.smithi138.stdout:1559: oid 47 version 261 is already newer than 261 2023-06-27T12:44:38.769 INFO:tasks.rados.rados.0.smithi138.stdout:1559: oid 47 version 261 is already newer than 261 2023-06-27T12:44:38.769 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 47 v 261 (ObjNum 747 snap 184 seq_num 747) dirty exists 2023-06-27T12:44:38.769 INFO:tasks.rados.rados.0.smithi138.stdout:1559: left oid 47 (ObjNum 747 snap 184 seq_num 747) 2023-06-27T12:44:38.779 INFO:tasks.rados.rados.0.smithi138.stdout:1560: expect (ObjNum 726 snap 178 seq_num 726) 2023-06-27T12:44:38.820 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:38.822 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:38.823 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:39.501 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:39.501 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:39.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:39.631 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:39.623+0000 7efdb2ff0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:39.633 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:39.629+0000 7efdb2ff0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:39.638 INFO:tasks.rados.rados.0.smithi138.stdout:1559: done (2 left) 2023-06-27T12:44:39.638 INFO:tasks.rados.rados.0.smithi138.stdout:1560: done (1 left) 2023-06-27T12:44:39.638 INFO:tasks.rados.rados.0.smithi138.stdout:1561: done (0 left) 2023-06-27T12:44:39.639 INFO:tasks.rados.rados.0.smithi138.stdout:1562: write oid 6 current snap is 185 2023-06-27T12:44:39.639 INFO:tasks.rados.rados.0.smithi138.stdout:1562: seq_num 748 ranges {571636=488846,1854531=565263} 2023-06-27T12:44:39.645 INFO:tasks.rados.rados.0.smithi138.stdout:1562: writing smithi138138488-6 from 571636 to 1060482 tid 1 2023-06-27T12:44:39.655 INFO:tasks.rados.rados.0.smithi138.stdout:1562: writing smithi138138488-6 from 1854531 to 2419794 tid 2 2023-06-27T12:44:39.655 INFO:tasks.rados.rados.0.smithi138.stdout:1563: read oid 20 snap -1 2023-06-27T12:44:39.655 INFO:tasks.rados.rados.0.smithi138.stdout:1563: expect (ObjNum 726 snap 178 seq_num 726) 2023-06-27T12:44:39.656 INFO:tasks.rados.rados.0.smithi138.stdout:1564: delete oid 45 current snap is 185 2023-06-27T12:44:39.699 INFO:tasks.rados.rados.0.smithi138.stdout:1563: expect (ObjNum 726 snap 178 seq_num 726) 2023-06-27T12:44:39.844 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:39.857 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":580,"stamp":"2023-06-27T12:44:39.633576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":106730093,"num_objects":394,"num_object_clones":347,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":2276,"num_read_kb":1686378,"num_write":2086,"num_write_kb":608051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3823,"ondisk_log_size":3823,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1161120,"kb_used_data":1025152,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376326240,"statfs":{"total":386547056640,"available":385358069760,"internally_reserved":0,"allocated":1049755648,"data_stored":1211001968,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8991943,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":20,"num_read_kb":13630,"num_write":16,"num_write_kb":4572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.151646"},"pg_stats":[{"pgid":"3.f","version":"361'54","reported_seq":907,"reported_epoch":383,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:37.630717+0000","last_change":"2023-06-27T12:44:37.634441+0000","last_active":"2023-06-27T12:44:37.630717+0000","last_peered":"2023-06-27T12:44:37.630717+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:37.630717+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3731286,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":13,"num_read_kb":7,"num_write":33,"num_write_kb":9288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"384'261","reported_seq":1214,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.769630+0000","last_change":"2023-06-27T12:44:38.652895+0000","last_active":"2023-06-27T12:44:38.769630+0000","last_peered":"2023-06-27T12:44:38.769630+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:38.769630+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12404160,"num_objects":33,"num_object_clones":29,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":168,"num_read_kb":208737,"num_write":180,"num_write_kb":54679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":33}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"376'315","reported_seq":1187,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.707302+0000","last_change":"2023-06-27T12:44:38.722607+0000","last_active":"2023-06-27T12:44:38.707302+0000","last_peered":"2023-06-27T12:44:38.707302+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:38.707302+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10622585,"num_objects":35,"num_object_clones":30,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":258,"num_read_kb":171536,"num_write":196,"num_write_kb":56915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":35}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"378'213","reported_seq":1127,"reported_epoch":383,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:37.632123+0000","last_change":"2023-06-27T12:44:37.661867+0000","last_active":"2023-06-27T12:44:37.632123+0000","last_peered":"2023-06-27T12:44:37.632123+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:37.632123+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8908139,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":197,"num_read_kb":141767,"num_write":139,"num_write_kb":39584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":30}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":651,"reported_epoch":383,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:37.624934+0000","last_change":"2023-06-27T12:44:37.634823+0000","last_active":"2023-06-27T12:44:37.624934+0000","last_peered":"2023-06-27T12:44:37.624934+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:37.624934+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"376'455","reported_seq":1215,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-06-27T12:44:37.636840+0000","last_change":"2023-06-27T12:44:37.636982+0000","last_active":"2023-06-27T12:44:37.636840+0000","last_peered":"2023-06-27T12:44:37.636840+0000","last_clean":"2023-06-27T12:44:37.636840+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:37.636840+0000","last_undegraded":"2023-06-27T12:44:37.636840+0000","last_fullsized":"2023-06-27T12:44:37.636840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10471388,"num_objects":49,"num_object_clones":43,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":333,"num_read_kb":211120,"num_write":265,"num_write_kb":75831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"382'238","reported_seq":1017,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-06-27T12:44:37.625886+0000","last_change":"2023-06-27T12:44:37.662599+0000","last_active":"2023-06-27T12:44:37.625886+0000","last_peered":"2023-06-27T12:44:37.625886+0000","last_clean":"2023-06-27T12:44:37.625886+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:37.625886+0000","last_undegraded":"2023-06-27T12:44:37.625886+0000","last_fullsized":"2023-06-27T12:44:37.625886+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4605448,"num_objects":29,"num_object_clones":26,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":139,"num_read_kb":85554,"num_write":143,"num_write_kb":41268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":946,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.704263+0000","last_change":"2023-06-27T12:44:38.709523+0000","last_active":"2023-06-27T12:44:38.704263+0000","last_peered":"2023-06-27T12:44:38.704263+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:38.704263+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":646,"reported_epoch":384,"state":"active+undersized","last_fresh":"2023-06-27T12:44:38.635623+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:38.635623+0000","last_peered":"2023-06-27T12:44:38.635623+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:38.635623+0000","last_undegraded":"2023-06-27T12:44:38.635623+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":854,"reported_epoch":384,"state":"active+undersized","last_fresh":"2023-06-27T12:44:38.704801+0000","last_change":"2023-06-27T12:44:38.721856+0000","last_active":"2023-06-27T12:44:38.704801+0000","last_peered":"2023-06-27T12:44:38.704801+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:38.704801+0000","last_undegraded":"2023-06-27T12:44:38.704801+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":623,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-06-27T12:44:37.632640+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:37.632640+0000","last_peered":"2023-06-27T12:44:37.632640+0000","last_clean":"2023-06-27T12:44:37.632640+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:37.632640+0000","last_undegraded":"2023-06-27T12:44:37.632640+0000","last_fullsized":"2023-06-27T12:44:37.632640+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"370'201","reported_seq":1089,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.653584+0000","last_change":"2023-06-27T12:44:38.653584+0000","last_active":"2023-06-27T12:44:38.653584+0000","last_peered":"2023-06-27T12:44:38.653584+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:38.653584+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4924748,"num_objects":23,"num_object_clones":18,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":85,"num_read_kb":75270,"num_write":128,"num_write_kb":35298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":612,"reported_epoch":384,"state":"active+undersized","last_fresh":"2023-06-27T12:44:38.711127+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:38.711127+0000","last_peered":"2023-06-27T12:44:38.711127+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:38.711127+0000","last_undegraded":"2023-06-27T12:44:38.711127+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"372'459","reported_seq":777,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.638350+0000","last_change":"2023-06-27T12:44:38.651880+0000","last_active":"2023-06-27T12:44:38.638350+0000","last_peered":"2023-06-27T12:44:38.638350+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:38.638350+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2890936,"num_objects":28,"num_object_clones":26,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":77,"num_read_kb":55296,"num_write":136,"num_write_kb":42509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":609,"reported_epoch":384,"state":"active+clean","last_fresh":"2023-06-27T12:44:38.706562+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:38.706562+0000","last_peered":"2023-06-27T12:44:38.706562+0000","last_clean":"2023-06-27T12:44:38.706562+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:38.706562+0000","last_undegraded":"2023-06-27T12:44:38.706562+0000","last_fullsized":"2023-06-27T12:44:38.706562+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":375,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-06-27T12:44:37.629039+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:37.629039+0000","last_peered":"2023-06-27T12:44:37.629039+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:37.629039+0000","last_undegraded":"2023-06-27T12:44:37.629039+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"380'447","reported_seq":1325,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.710147+0000","last_change":"2023-06-27T12:44:38.721671+0000","last_active":"2023-06-27T12:44:38.710147+0000","last_peered":"2023-06-27T12:44:38.710147+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:38.710147+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11143668,"num_objects":54,"num_object_clones":48,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":327,"num_read_kb":198257,"num_write":304,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":54}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":623,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-06-27T12:44:37.630638+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:37.630638+0000","last_peered":"2023-06-27T12:44:37.630638+0000","last_clean":"2023-06-27T12:44:37.630638+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:37.630638+0000","last_undegraded":"2023-06-27T12:44:37.630638+0000","last_fullsized":"2023-06-27T12:44:37.630638+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"373'271","reported_seq":711,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.713514+0000","last_change":"2023-06-27T12:44:38.719859+0000","last_active":"2023-06-27T12:44:38.713514+0000","last_peered":"2023-06-27T12:44:38.713514+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:38.713514+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8925012,"num_objects":20,"num_object_clones":20,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":73338,"num_write":117,"num_write_kb":33414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":389,"reported_epoch":384,"state":"active+undersized","last_fresh":"2023-06-27T12:44:38.640996+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:38.640996+0000","last_peered":"2023-06-27T12:44:38.640996+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:38.640996+0000","last_undegraded":"2023-06-27T12:44:38.640996+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"382'238","reported_seq":1149,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.642502+0000","last_change":"2023-06-27T12:44:38.661099+0000","last_active":"2023-06-27T12:44:38.642502+0000","last_peered":"2023-06-27T12:44:38.642502+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:38.642502+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10673039,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":142738,"num_write":154,"num_write_kb":44067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"382'155","reported_seq":657,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.716576+0000","last_change":"2023-06-27T12:44:38.737331+0000","last_active":"2023-06-27T12:44:38.716576+0000","last_peered":"2023-06-27T12:44:38.716576+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:38.716576+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420373,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":62857,"num_write":62,"num_write_kb":17546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":646,"reported_epoch":384,"state":"active+undersized","last_fresh":"2023-06-27T12:44:38.639949+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:38.639949+0000","last_peered":"2023-06-27T12:44:38.639949+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:38.639949+0000","last_undegraded":"2023-06-27T12:44:38.639949+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":609,"reported_epoch":384,"state":"active+clean","last_fresh":"2023-06-27T12:44:38.702270+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:38.702270+0000","last_peered":"2023-06-27T12:44:38.702270+0000","last_clean":"2023-06-27T12:44:38.702270+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:38.702270+0000","last_undegraded":"2023-06-27T12:44:38.702270+0000","last_fullsized":"2023-06-27T12:44:38.702270+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"361'236","reported_seq":1127,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:38.636829+0000","last_change":"2023-06-27T12:44:38.646319+0000","last_active":"2023-06-27T12:44:38.636829+0000","last_peered":"2023-06-27T12:44:38.636829+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:38.636829+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5916328,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":124,"num_read_kb":135931,"num_write":125,"num_write_kb":37073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":106730093,"num_objects":394,"num_object_clones":347,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":2276,"num_read_kb":1686378,"num_write":2086,"num_write_kb":608051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3726,"ondisk_log_size":3726,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673365,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308348,"kb_used_data":273332,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94063492,"statfs":{"total":96636764160,"available":96321015808,"internally_reserved":0,"allocated":279891968,"data_stored":325792958,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":321624,"kb_used_data":286608,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94050216,"statfs":{"total":96636764160,"available":96307421184,"internally_reserved":0,"allocated":293486592,"data_stored":335445224,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673365,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":349868,"kb_used_data":314852,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94021972,"statfs":{"total":96636764160,"available":96278499328,"internally_reserved":0,"allocated":322408448,"data_stored":372724234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:39.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.e 3.c 3.b 3.0 3.1 3.4 3.6 2023-06-27T12:44:39.897 INFO:tasks.rados.rados.0.smithi138.stdout:1562: finishing write tid 1 to smithi138138488-6 2023-06-27T12:44:39.897 INFO:tasks.rados.rados.0.smithi138.stdout:1562: finishing write tid 2 to smithi138138488-6 2023-06-27T12:44:39.897 INFO:tasks.rados.rados.0.smithi138.stdout:1562: finishing write tid 3 to smithi138138488-6 2023-06-27T12:44:39.898 INFO:tasks.rados.rados.0.smithi138.stdout:1563: done (2 left) 2023-06-27T12:44:39.898 INFO:tasks.rados.rados.0.smithi138.stdout:1564: done (1 left) 2023-06-27T12:44:39.898 INFO:tasks.rados.rados.0.smithi138.stdout:1565: write (excl) oid 2 current snap is 185 2023-06-27T12:44:39.898 INFO:tasks.rados.rados.0.smithi138.stdout:1562: finishing write tid 4 to smithi138138488-6 2023-06-27T12:44:39.898 INFO:tasks.rados.rados.0.smithi138.stdout:1562: oid 6 updating version 0 to 206 2023-06-27T12:44:39.898 INFO:tasks.rados.rados.0.smithi138.stdout:1562: oid 6 version 206 is already newer than 206 2023-06-27T12:44:39.898 INFO:tasks.rados.rados.0.smithi138.stdout:1562: oid 6 version 206 is already newer than 206 2023-06-27T12:44:39.899 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 206 (ObjNum 748 snap 185 seq_num 748) dirty exists 2023-06-27T12:44:39.899 INFO:tasks.rados.rados.0.smithi138.stdout:1562: left oid 6 (ObjNum 748 snap 185 seq_num 748) 2023-06-27T12:44:39.899 INFO:tasks.rados.rados.0.smithi138.stdout:1565: seq_num 750 ranges {605250=639344,1855142=452579} 2023-06-27T12:44:39.904 INFO:tasks.rados.rados.0.smithi138.stdout:1565: writing smithi138138488-2 from 605250 to 1244594 tid 1 2023-06-27T12:44:39.910 INFO:tasks.rados.rados.0.smithi138.stdout:1565: writing smithi138138488-2 from 1855142 to 2307721 tid 2 2023-06-27T12:44:39.911 INFO:tasks.rados.rados.0.smithi138.stdout:1562: done (1 left) 2023-06-27T12:44:39.911 INFO:tasks.rados.rados.0.smithi138.stdout:1566: snap_remove snap 181 2023-06-27T12:44:39.999 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:39.985+0000 7f36ba80a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:39.998+0000 7f36ba80a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:40.199 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.c] on osd.0 to force-backfill; instructing pg(s) [3.1] on osd.1 to force-backfill; instructing pg(s) [3.0,3.4,3.6,3.e] on osd.2 to force-backfill; 2023-06-27T12:44:40.199 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:44:40.464 INFO:tasks.rados.rados.0.smithi138.stdout:1566: done (1 left) 2023-06-27T12:44:40.465 INFO:tasks.rados.rados.0.smithi138.stdout:1567: write oid 28 current snap is 185 2023-06-27T12:44:40.465 INFO:tasks.rados.rados.0.smithi138.stdout:1565: finishing write tid 1 to smithi138138488-2 2023-06-27T12:44:40.465 INFO:tasks.rados.rados.0.smithi138.stdout:1567: seq_num 751 ranges {622057=714593,1913624=708433} 2023-06-27T12:44:40.474 INFO:tasks.rados.rados.0.smithi138.stdout:1567: writing smithi138138488-28 from 622057 to 1336650 tid 1 2023-06-27T12:44:40.485 INFO:tasks.rados.rados.0.smithi138.stdout:1567: writing smithi138138488-28 from 1913624 to 2622057 tid 2 2023-06-27T12:44:40.486 INFO:tasks.rados.rados.0.smithi138.stdout:1568: write (excl) oid 42 current snap is 185 2023-06-27T12:44:40.486 INFO:tasks.rados.rados.0.smithi138.stdout:1565: finishing write tid 2 to smithi138138488-2 2023-06-27T12:44:40.486 INFO:tasks.rados.rados.0.smithi138.stdout:1565: finishing write tid 3 to smithi138138488-2 2023-06-27T12:44:40.486 INFO:tasks.rados.rados.0.smithi138.stdout:1568: seq_num 752 ranges {638864=789842,1972106=456248} 2023-06-27T12:44:40.496 INFO:tasks.rados.rados.0.smithi138.stdout:1568: writing smithi138138488-42 from 638864 to 1428706 tid 1 2023-06-27T12:44:40.504 INFO:tasks.rados.rados.0.smithi138.stdout:1568: writing smithi138138488-42 from 1972106 to 2428354 tid 2 2023-06-27T12:44:40.504 INFO:tasks.rados.rados.0.smithi138.stdout:1569: snap_remove snap 179 2023-06-27T12:44:40.824 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:40.952 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:40.947+0000 7f86ee391700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:40.955 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:40.951+0000 7f86ee391700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:41.001 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:41.003 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:41.004 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:41.470 INFO:tasks.rados.rados.0.smithi138.stdout:1569: done (3 left) 2023-06-27T12:44:41.471 INFO:tasks.rados.rados.0.smithi138.stdout:1570: read oid 35 snap -1 2023-06-27T12:44:41.471 INFO:tasks.rados.rados.0.smithi138.stdout:1570: expect (ObjNum 676 snap 167 seq_num 676) 2023-06-27T12:44:41.471 INFO:tasks.rados.rados.0.smithi138.stdout:1571: write (excl) oid 38 current snap is 185 2023-06-27T12:44:41.471 INFO:tasks.rados.rados.0.smithi138.stdout:1565: finishing write tid 4 to smithi138138488-2 2023-06-27T12:44:41.472 INFO:tasks.rados.rados.0.smithi138.stdout:1565: oid 2 updating version 0 to 477 2023-06-27T12:44:41.472 INFO:tasks.rados.rados.0.smithi138.stdout:1565: oid 2 updating version 477 to 479 2023-06-27T12:44:41.472 INFO:tasks.rados.rados.0.smithi138.stdout:1565: oid 2 version 479 is already newer than 479 2023-06-27T12:44:41.472 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 479 (ObjNum 750 snap 185 seq_num 750) dirty exists 2023-06-27T12:44:41.472 INFO:tasks.rados.rados.0.smithi138.stdout:1565: left oid 2 (ObjNum 750 snap 185 seq_num 750) 2023-06-27T12:44:41.472 INFO:tasks.rados.rados.0.smithi138.stdout:1567: finishing write tid 1 to smithi138138488-28 2023-06-27T12:44:41.472 INFO:tasks.rados.rados.0.smithi138.stdout:1571: seq_num 753 ranges {655671=581444,2030588=625083} 2023-06-27T12:44:41.478 INFO:tasks.rados.rados.0.smithi138.stdout:1571: writing smithi138138488-38 from 655671 to 1237115 tid 1 2023-06-27T12:44:41.489 INFO:tasks.rados.rados.0.smithi138.stdout:1571: writing smithi138138488-38 from 2030588 to 2655671 tid 2 2023-06-27T12:44:41.490 INFO:tasks.rados.rados.0.smithi138.stdout:1565: done (4 left) 2023-06-27T12:44:41.490 INFO:tasks.rados.rados.0.smithi138.stdout:1572: snap_remove snap 184 2023-06-27T12:44:42.470 INFO:tasks.rados.rados.0.smithi138.stdout:1572: done (4 left) 2023-06-27T12:44:42.471 INFO:tasks.rados.rados.0.smithi138.stdout:1573: snap_create 2023-06-27T12:44:42.471 INFO:tasks.rados.rados.0.smithi138.stdout:1568: finishing write tid 1 to smithi138138488-42 2023-06-27T12:44:42.471 INFO:tasks.rados.rados.0.smithi138.stdout:1567: finishing write tid 2 to smithi138138488-28 2023-06-27T12:44:42.471 INFO:tasks.rados.rados.0.smithi138.stdout:1567: finishing write tid 3 to smithi138138488-28 2023-06-27T12:44:42.471 INFO:tasks.rados.rados.0.smithi138.stdout:1567: finishing write tid 4 to smithi138138488-28 2023-06-27T12:44:42.471 INFO:tasks.rados.rados.0.smithi138.stdout:1567: oid 28 updating version 0 to 253 2023-06-27T12:44:42.472 INFO:tasks.rados.rados.0.smithi138.stdout:1567: oid 28 version 253 is already newer than 253 2023-06-27T12:44:42.472 INFO:tasks.rados.rados.0.smithi138.stdout:1567: oid 28 version 253 is already newer than 247 2023-06-27T12:44:42.472 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 253 (ObjNum 751 snap 185 seq_num 751) dirty exists 2023-06-27T12:44:42.472 INFO:tasks.rados.rados.0.smithi138.stdout:1567: left oid 28 (ObjNum 751 snap 185 seq_num 751) 2023-06-27T12:44:42.472 INFO:tasks.rados.rados.0.smithi138.stdout:1568: finishing write tid 2 to smithi138138488-42 2023-06-27T12:44:42.472 INFO:tasks.rados.rados.0.smithi138.stdout:1568: finishing write tid 3 to smithi138138488-42 2023-06-27T12:44:42.472 INFO:tasks.rados.rados.0.smithi138.stdout:1568: finishing write tid 4 to smithi138138488-42 2023-06-27T12:44:42.472 INFO:tasks.rados.rados.0.smithi138.stdout:1568: oid 42 updating version 0 to 216 2023-06-27T12:44:42.473 INFO:tasks.rados.rados.0.smithi138.stdout:1568: oid 42 updating version 216 to 219 2023-06-27T12:44:42.473 INFO:tasks.rados.rados.0.smithi138.stdout:1568: oid 42 version 219 is already newer than 219 2023-06-27T12:44:42.473 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 219 (ObjNum 752 snap 185 seq_num 752) dirty exists 2023-06-27T12:44:42.473 INFO:tasks.rados.rados.0.smithi138.stdout:1568: left oid 42 (ObjNum 752 snap 185 seq_num 752) 2023-06-27T12:44:42.473 INFO:tasks.rados.rados.0.smithi138.stdout:1570: expect (ObjNum 676 snap 167 seq_num 676) 2023-06-27T12:44:42.815 INFO:tasks.rados.rados.0.smithi138.stdout:1571: finishing write tid 1 to smithi138138488-38 2023-06-27T12:44:42.816 INFO:tasks.rados.rados.0.smithi138.stdout:1571: finishing write tid 2 to smithi138138488-38 2023-06-27T12:44:42.816 INFO:tasks.rados.rados.0.smithi138.stdout:1571: finishing write tid 3 to smithi138138488-38 2023-06-27T12:44:42.816 INFO:tasks.rados.rados.0.smithi138.stdout:1571: finishing write tid 4 to smithi138138488-38 2023-06-27T12:44:42.816 INFO:tasks.rados.rados.0.smithi138.stdout:1571: oid 38 updating version 0 to 58 2023-06-27T12:44:42.816 INFO:tasks.rados.rados.0.smithi138.stdout:1571: oid 38 version 58 is already newer than 58 2023-06-27T12:44:42.816 INFO:tasks.rados.rados.0.smithi138.stdout:1571: oid 38 version 58 is already newer than 55 2023-06-27T12:44:42.817 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 58 (ObjNum 753 snap 185 seq_num 753) dirty exists 2023-06-27T12:44:42.817 INFO:tasks.rados.rados.0.smithi138.stdout:1571: left oid 38 (ObjNum 753 snap 185 seq_num 753) 2023-06-27T12:44:42.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~248s 2023-06-27T12:44:43.005 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:43.130 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:43.125+0000 7f2eff799700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:43.133 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:43.129+0000 7f2eff799700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:43.181 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:43.182 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:43.474 INFO:tasks.rados.rados.0.smithi138.stdout:1567: done (4 left) 2023-06-27T12:44:43.474 INFO:tasks.rados.rados.0.smithi138.stdout:1568: done (3 left) 2023-06-27T12:44:43.475 INFO:tasks.rados.rados.0.smithi138.stdout:1570: done (2 left) 2023-06-27T12:44:43.475 INFO:tasks.rados.rados.0.smithi138.stdout:1571: done (1 left) 2023-06-27T12:44:43.475 INFO:tasks.rados.rados.0.smithi138.stdout:1573: done (0 left) 2023-06-27T12:44:43.475 INFO:tasks.rados.rados.0.smithi138.stdout:1574: read oid 11 snap 182 2023-06-27T12:44:43.475 INFO:tasks.rados.rados.0.smithi138.stdout:1574: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:44:43.475 INFO:tasks.rados.rados.0.smithi138.stdout:1575: snap_create 2023-06-27T12:44:43.502 INFO:tasks.rados.rados.0.smithi138.stdout:1574: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:44:44.474 INFO:tasks.rados.rados.0.smithi138.stdout:1574: done (1 left) 2023-06-27T12:44:44.475 INFO:tasks.rados.rados.0.smithi138.stdout:1575: done (0 left) 2023-06-27T12:44:44.475 INFO:tasks.rados.rados.0.smithi138.stdout:1576: write oid 50 current snap is 187 2023-06-27T12:44:44.475 INFO:tasks.rados.rados.0.smithi138.stdout:1576: seq_num 754 ranges {672478=656693,2089070=459917} 2023-06-27T12:44:44.484 INFO:tasks.rados.rados.0.smithi138.stdout:1576: writing smithi138138488-50 from 672478 to 1329171 tid 1 2023-06-27T12:44:44.493 INFO:tasks.rados.rados.0.smithi138.stdout:1576: writing smithi138138488-50 from 2089070 to 2548987 tid 2 2023-06-27T12:44:44.493 INFO:tasks.rados.rados.0.smithi138.stdout:1577: delete oid 28 current snap is 187 2023-06-27T12:44:44.555 INFO:tasks.rados.rados.0.smithi138.stdout:1576: finishing write tid 1 to smithi138138488-50 2023-06-27T12:44:44.559 INFO:tasks.rados.rados.0.smithi138.stdout:1576: finishing write tid 2 to smithi138138488-50 2023-06-27T12:44:44.600 INFO:tasks.rados.rados.0.smithi138.stdout:1576: finishing write tid 3 to smithi138138488-50 2023-06-27T12:44:44.601 INFO:tasks.rados.rados.0.smithi138.stdout:1577: done (1 left) 2023-06-27T12:44:44.601 INFO:tasks.rados.rados.0.smithi138.stdout:1578: delete oid 27 current snap is 187 2023-06-27T12:44:44.602 INFO:tasks.rados.rados.0.smithi138.stdout:1576: finishing write tid 4 to smithi138138488-50 2023-06-27T12:44:44.602 INFO:tasks.rados.rados.0.smithi138.stdout:1576: oid 50 updating version 0 to 83 2023-06-27T12:44:44.602 INFO:tasks.rados.rados.0.smithi138.stdout:1576: oid 50 version 83 is already newer than 83 2023-06-27T12:44:44.602 INFO:tasks.rados.rados.0.smithi138.stdout:1576: oid 50 version 83 is already newer than 83 2023-06-27T12:44:44.603 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 83 (ObjNum 754 snap 187 seq_num 754) dirty exists 2023-06-27T12:44:44.603 INFO:tasks.rados.rados.0.smithi138.stdout:1576: left oid 50 (ObjNum 754 snap 187 seq_num 754) 2023-06-27T12:44:44.650 INFO:tasks.rados.rados.0.smithi138.stdout:1576: done (1 left) 2023-06-27T12:44:44.650 INFO:tasks.rados.rados.0.smithi138.stdout:1578: done (0 left) 2023-06-27T12:44:44.650 INFO:tasks.rados.rados.0.smithi138.stdout:1579: delete oid 30 current snap is 187 2023-06-27T12:44:44.701 INFO:tasks.rados.rados.0.smithi138.stdout:1579: done (0 left) 2023-06-27T12:44:44.701 INFO:tasks.rados.rados.0.smithi138.stdout:1580: read oid 36 snap -1 2023-06-27T12:44:44.701 INFO:tasks.rados.rados.0.smithi138.stdout:1580: expect (ObjNum 700 snap 171 seq_num 700) 2023-06-27T12:44:44.702 INFO:tasks.rados.rados.0.smithi138.stdout:1581: read oid 1 snap -1 2023-06-27T12:44:44.702 INFO:tasks.rados.rados.0.smithi138.stdout:1581: expect (ObjNum 742 snap 182 seq_num 742) 2023-06-27T12:44:44.702 INFO:tasks.rados.rados.0.smithi138.stdout:1582: write (excl) oid 46 current snap is 187 2023-06-27T12:44:44.702 INFO:tasks.rados.rados.0.smithi138.stdout:1582: seq_num 758 ranges {739706=557689,1806645=467255} 2023-06-27T12:44:44.710 INFO:tasks.rados.rados.0.smithi138.stdout:1582: writing smithi138138488-46 from 739706 to 1297395 tid 1 2023-06-27T12:44:44.718 INFO:tasks.rados.rados.0.smithi138.stdout:1582: writing smithi138138488-46 from 1806645 to 2273900 tid 2 2023-06-27T12:44:44.718 INFO:tasks.rados.rados.0.smithi138.stdout:1583: snap_create 2023-06-27T12:44:44.747 INFO:tasks.rados.rados.0.smithi138.stdout:1580: expect (ObjNum 700 snap 171 seq_num 700) 2023-06-27T12:44:44.982 INFO:tasks.rados.rados.0.smithi138.stdout:1581: expect (ObjNum 742 snap 182 seq_num 742) 2023-06-27T12:44:45.073 INFO:tasks.rados.rados.0.smithi138.stdout:1582: finishing write tid 1 to smithi138138488-46 2023-06-27T12:44:45.073 INFO:tasks.rados.rados.0.smithi138.stdout:1582: finishing write tid 2 to smithi138138488-46 2023-06-27T12:44:45.073 INFO:tasks.rados.rados.0.smithi138.stdout:1582: finishing write tid 3 to smithi138138488-46 2023-06-27T12:44:45.073 INFO:tasks.rados.rados.0.smithi138.stdout:1582: finishing write tid 4 to smithi138138488-46 2023-06-27T12:44:45.073 INFO:tasks.rados.rados.0.smithi138.stdout:1582: oid 46 updating version 0 to 214 2023-06-27T12:44:45.074 INFO:tasks.rados.rados.0.smithi138.stdout:1582: oid 46 version 214 is already newer than 211 2023-06-27T12:44:45.074 INFO:tasks.rados.rados.0.smithi138.stdout:1582: oid 46 version 214 is already newer than 214 2023-06-27T12:44:45.074 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 46 v 214 (ObjNum 758 snap 187 seq_num 758) dirty exists 2023-06-27T12:44:45.074 INFO:tasks.rados.rados.0.smithi138.stdout:1582: left oid 46 (ObjNum 758 snap 187 seq_num 758) 2023-06-27T12:44:45.183 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:45.211 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:45.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:45.211 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:45.313 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:45.308+0000 7f1940338700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:45.316 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:45.312+0000 7f1940338700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:45.353+0000 7f7bdfc90700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:45.364 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:45.360+0000 7f7bdfc90700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:45.367 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:45.371 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:45.371 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:45.493 INFO:tasks.rados.rados.0.smithi138.stdout:1580: done (3 left) 2023-06-27T12:44:45.493 INFO:tasks.rados.rados.0.smithi138.stdout:1581: done (2 left) 2023-06-27T12:44:45.493 INFO:tasks.rados.rados.0.smithi138.stdout:1582: done (1 left) 2023-06-27T12:44:45.493 INFO:tasks.rados.rados.0.smithi138.stdout:1583: done (0 left) 2023-06-27T12:44:45.493 INFO:tasks.rados.rados.0.smithi138.stdout:1584: write (excl) oid 18 current snap is 188 2023-06-27T12:44:45.493 INFO:tasks.rados.rados.0.smithi138.stdout:1584: seq_num 759 ranges {756513=632938,1865127=727266} 2023-06-27T12:44:45.502 INFO:tasks.rados.rados.0.smithi138.stdout:1584: writing smithi138138488-18 from 756513 to 1389451 tid 1 2023-06-27T12:44:45.512 INFO:tasks.rados.rados.0.smithi138.stdout:1584: writing smithi138138488-18 from 1865127 to 2592393 tid 2 2023-06-27T12:44:45.513 INFO:tasks.rados.rados.0.smithi138.stdout:1585: snap_create 2023-06-27T12:44:45.562 INFO:tasks.rados.rados.0.smithi138.stdout:1584: finishing write tid 1 to smithi138138488-18 2023-06-27T12:44:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:45.566 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:45.579 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":589,"stamp":"2023-06-27T12:44:45.488256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110797783,"num_objects":397,"num_object_clones":350,"num_object_copies":1191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":16,"num_read":2296,"num_read_kb":1701028,"num_write":2106,"num_write_kb":613936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3873,"ondisk_log_size":3873,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1173104,"kb_used_data":1037136,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376314256,"statfs":{"total":386547056640,"available":385345798144,"internally_reserved":0,"allocated":1062027264,"data_stored":1224203652,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4067690,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":20,"num_read_kb":14650,"num_write":20,"num_write_kb":5885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024041"},"pg_stats":[{"pgid":"3.f","version":"387'58","reported_seq":920,"reported_epoch":387,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:41.571191+0000","last_change":"2023-06-27T12:44:41.508781+0000","last_active":"2023-06-27T12:44:41.571191+0000","last_peered":"2023-06-27T12:44:41.571191+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:41.571191+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2655671,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":15,"num_read_kb":8,"num_write":37,"num_write_kb":10467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"384'261","reported_seq":1227,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.496240+0000","last_change":"2023-06-27T12:44:43.501091+0000","last_active":"2023-06-27T12:44:43.496240+0000","last_peered":"2023-06-27T12:44:43.496240+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:43.496240+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12404160,"num_objects":33,"num_object_clones":29,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":213039,"num_write":180,"num_write_kb":54679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":33}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"376'315","reported_seq":1203,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.496395+0000","last_change":"2023-06-27T12:44:42.487645+0000","last_active":"2023-06-27T12:44:43.496395+0000","last_peered":"2023-06-27T12:44:43.496395+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:43.496395+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10622585,"num_objects":35,"num_object_clones":30,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":265,"num_read_kb":176835,"num_write":196,"num_write_kb":56915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":35}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"386'219","reported_seq":1140,"reported_epoch":387,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:41.500978+0000","last_change":"2023-06-27T12:44:41.521194+0000","last_active":"2023-06-27T12:44:41.500978+0000","last_peered":"2023-06-27T12:44:41.500978+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:41.500978+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9143437,"num_objects":31,"num_object_clones":27,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":199,"num_read_kb":141768,"num_write":143,"num_write_kb":40802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":31}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":660,"reported_epoch":387,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:41.492274+0000","last_change":"2023-06-27T12:44:41.506783+0000","last_active":"2023-06-27T12:44:41.492274+0000","last_peered":"2023-06-27T12:44:41.492274+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:41.492274+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"376'455","reported_seq":1224,"reported_epoch":387,"state":"active+clean","last_fresh":"2023-06-27T12:44:41.496935+0000","last_change":"2023-06-27T12:44:41.507586+0000","last_active":"2023-06-27T12:44:41.496935+0000","last_peered":"2023-06-27T12:44:41.496935+0000","last_clean":"2023-06-27T12:44:41.496935+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:41.496935+0000","last_undegraded":"2023-06-27T12:44:41.496935+0000","last_fullsized":"2023-06-27T12:44:41.496935+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10471388,"num_objects":49,"num_object_clones":43,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":333,"num_read_kb":211120,"num_write":265,"num_write_kb":75831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"386'253","reported_seq":1031,"reported_epoch":387,"state":"active+clean","last_fresh":"2023-06-27T12:44:41.568908+0000","last_change":"2023-06-27T12:44:41.569094+0000","last_active":"2023-06-27T12:44:41.568908+0000","last_peered":"2023-06-27T12:44:41.568908+0000","last_clean":"2023-06-27T12:44:41.568908+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:41.568908+0000","last_undegraded":"2023-06-27T12:44:41.568908+0000","last_fullsized":"2023-06-27T12:44:41.568908+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7227505,"num_objects":29,"num_object_clones":26,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":85555,"num_write":147,"num_write_kb":42658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"336'79","reported_seq":959,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.485082+0000","last_change":"2023-06-27T12:44:42.484221+0000","last_active":"2023-06-27T12:44:43.485082+0000","last_peered":"2023-06-27T12:44:43.485082+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:43.485082+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3008585,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":47128,"num_write":46,"num_write_kb":13954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":654,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-06-27T12:44:43.488841+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:43.488841+0000","last_peered":"2023-06-27T12:44:43.488841+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:43.488841+0000","last_undegraded":"2023-06-27T12:44:43.488841+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":867,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-06-27T12:44:43.487304+0000","last_change":"2023-06-27T12:44:42.487128+0000","last_active":"2023-06-27T12:44:43.487304+0000","last_peered":"2023-06-27T12:44:43.487304+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:43.487304+0000","last_undegraded":"2023-06-27T12:44:43.487304+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":629,"reported_epoch":387,"state":"active+clean","last_fresh":"2023-06-27T12:44:41.498441+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:41.498441+0000","last_peered":"2023-06-27T12:44:41.498441+0000","last_clean":"2023-06-27T12:44:41.498441+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:41.498441+0000","last_undegraded":"2023-06-27T12:44:41.498441+0000","last_fullsized":"2023-06-27T12:44:41.498441+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"385'206","reported_seq":1104,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.493102+0000","last_change":"2023-06-27T12:44:43.500565+0000","last_active":"2023-06-27T12:44:43.493102+0000","last_peered":"2023-06-27T12:44:43.493102+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:43.493102+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7496384,"num_objects":23,"num_object_clones":18,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":86,"num_read_kb":75271,"num_write":132,"num_write_kb":36329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":23}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":622,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-06-27T12:44:43.484217+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:43.484217+0000","last_peered":"2023-06-27T12:44:43.484217+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:43.484217+0000","last_undegraded":"2023-06-27T12:44:43.484217+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"385'479","reported_seq":795,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.480727+0000","last_change":"2023-06-27T12:44:43.548447+0000","last_active":"2023-06-27T12:44:43.480727+0000","last_peered":"2023-06-27T12:44:43.480727+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:43.480727+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2605250,"num_objects":29,"num_object_clones":27,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":82,"num_read_kb":60341,"num_write":140,"num_write_kb":43576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":29}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":619,"reported_epoch":389,"state":"active+clean","last_fresh":"2023-06-27T12:44:43.484466+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:43.484466+0000","last_peered":"2023-06-27T12:44:43.484466+0000","last_clean":"2023-06-27T12:44:43.484466+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:43.484466+0000","last_undegraded":"2023-06-27T12:44:43.484466+0000","last_fullsized":"2023-06-27T12:44:43.484466+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":381,"reported_epoch":387,"state":"active+undersized","last_fresh":"2023-06-27T12:44:41.498546+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:41.498546+0000","last_peered":"2023-06-27T12:44:41.498546+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:41.498546+0000","last_undegraded":"2023-06-27T12:44:41.498546+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"380'447","reported_seq":1338,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.482110+0000","last_change":"2023-06-27T12:44:42.483638+0000","last_active":"2023-06-27T12:44:43.482110+0000","last_peered":"2023-06-27T12:44:43.482110+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:43.482110+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11143668,"num_objects":54,"num_object_clones":48,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":327,"num_read_kb":198257,"num_write":304,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":54}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":629,"reported_epoch":387,"state":"active+clean","last_fresh":"2023-06-27T12:44:41.497932+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:41.497932+0000","last_peered":"2023-06-27T12:44:41.497932+0000","last_clean":"2023-06-27T12:44:41.497932+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:41.497932+0000","last_undegraded":"2023-06-27T12:44:41.497932+0000","last_fullsized":"2023-06-27T12:44:41.497932+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"373'271","reported_seq":724,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.486312+0000","last_change":"2023-06-27T12:44:42.488208+0000","last_active":"2023-06-27T12:44:43.486312+0000","last_peered":"2023-06-27T12:44:43.486312+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:43.486312+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8925012,"num_objects":20,"num_object_clones":20,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":73338,"num_write":117,"num_write_kb":33414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":397,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-06-27T12:44:43.489203+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:43.489203+0000","last_peered":"2023-06-27T12:44:43.489203+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:43.489203+0000","last_undegraded":"2023-06-27T12:44:43.489203+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"382'238","reported_seq":1160,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.490977+0000","last_change":"2023-06-27T12:44:43.497197+0000","last_active":"2023-06-27T12:44:43.490977+0000","last_peered":"2023-06-27T12:44:43.490977+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:43.490977+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10673039,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":142738,"num_write":154,"num_write_kb":44067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"382'155","reported_seq":670,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.485377+0000","last_change":"2023-06-27T12:44:42.484303+0000","last_active":"2023-06-27T12:44:43.485377+0000","last_peered":"2023-06-27T12:44:43.485377+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:43.485377+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420373,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":62857,"num_write":62,"num_write_kb":17546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":654,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-06-27T12:44:43.490769+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:43.490769+0000","last_peered":"2023-06-27T12:44:43.490769+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:43.490769+0000","last_undegraded":"2023-06-27T12:44:43.490769+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":619,"reported_epoch":389,"state":"active+clean","last_fresh":"2023-06-27T12:44:43.483772+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:43.483772+0000","last_peered":"2023-06-27T12:44:43.483772+0000","last_clean":"2023-06-27T12:44:43.483772+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:43.483772+0000","last_undegraded":"2023-06-27T12:44:43.483772+0000","last_fullsized":"2023-06-27T12:44:43.483772+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"361'236","reported_seq":1138,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:43.492328+0000","last_change":"2023-06-27T12:44:43.495558+0000","last_active":"2023-06-27T12:44:43.492328+0000","last_peered":"2023-06-27T12:44:43.492328+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:43.492328+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5916328,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":124,"num_read_kb":135931,"num_write":125,"num_write_kb":37073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":110797783,"num_objects":397,"num_object_clones":350,"num_object_copies":1191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":16,"num_read":2296,"num_read_kb":1701028,"num_write":2106,"num_write_kb":613936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3776,"ondisk_log_size":3776,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673370,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":313124,"kb_used_data":278108,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94058716,"statfs":{"total":96636764160,"available":96316125184,"internally_reserved":0,"allocated":284782592,"data_stored":331370485,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706075,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":325816,"kb_used_data":290800,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94046024,"statfs":{"total":96636764160,"available":96303128576,"internally_reserved":0,"allocated":297779200,"data_stored":340228933,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673370,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":352884,"kb_used_data":317868,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94018956,"statfs":{"total":96636764160,"available":96275410944,"internally_reserved":0,"allocated":325496832,"data_stored":375564682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:45.581 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.e 3.c 3.8 3.2 3.7 2023-06-27T12:44:45.603 INFO:tasks.rados.rados.0.smithi138.stdout:1584: finishing write tid 2 to smithi138138488-18 2023-06-27T12:44:45.603 INFO:tasks.rados.rados.0.smithi138.stdout:1584: finishing write tid 3 to smithi138138488-18 2023-06-27T12:44:45.603 INFO:tasks.rados.rados.0.smithi138.stdout:1584: finishing write tid 4 to smithi138138488-18 2023-06-27T12:44:45.603 INFO:tasks.rados.rados.0.smithi138.stdout:1584: oid 18 updating version 0 to 320 2023-06-27T12:44:45.603 INFO:tasks.rados.rados.0.smithi138.stdout:1584: oid 18 version 320 is already newer than 320 2023-06-27T12:44:45.604 INFO:tasks.rados.rados.0.smithi138.stdout:1584: oid 18 version 320 is already newer than 318 2023-06-27T12:44:45.604 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 320 (ObjNum 759 snap 188 seq_num 759) dirty exists 2023-06-27T12:44:45.604 INFO:tasks.rados.rados.0.smithi138.stdout:1584: left oid 18 (ObjNum 759 snap 188 seq_num 759) 2023-06-27T12:44:45.713 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:45.701+0000 7f78ac6dc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:45.716 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:45.712+0000 7f78ac6dc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:45.928 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c] on osd.0 to force-backfill; instructing pg(s) [3.2,3.7,3.8] on osd.1 to force-backfill; instructing pg(s) [3.e] on osd.2 to force-backfill; 2023-06-27T12:44:45.928 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:44:46.499 INFO:tasks.rados.rados.0.smithi138.stdout:1584: done (1 left) 2023-06-27T12:44:46.499 INFO:tasks.rados.rados.0.smithi138.stdout:1585: done (0 left) 2023-06-27T12:44:46.500 INFO:tasks.rados.rados.0.smithi138.stdout:1586: snap_remove snap 185 2023-06-27T12:44:47.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:47.500 INFO:tasks.rados.rados.0.smithi138.stdout:1586: done (0 left) 2023-06-27T12:44:47.500 INFO:tasks.rados.rados.0.smithi138.stdout:1587: delete oid 28 current snap is 189 2023-06-27T12:44:47.513 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:47.508+0000 7fb1b94a7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:47.516 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:47.512+0000 7fb1b94a7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:47.527 INFO:tasks.rados.rados.0.smithi138.stdout:1587: done (0 left) 2023-06-27T12:44:47.528 INFO:tasks.rados.rados.0.smithi138.stdout:1588: write oid 17 current snap is 189 2023-06-27T12:44:47.528 INFO:tasks.rados.rados.0.smithi138.stdout:1588: seq_num 761 ranges {790127=499789,1698444=730935} 2023-06-27T12:44:47.537 INFO:tasks.rados.rados.0.smithi138.stdout:1588: writing smithi138138488-17 from 790127 to 1289916 tid 1 2023-06-27T12:44:47.547 INFO:tasks.rados.rados.0.smithi138.stdout:1588: writing smithi138138488-17 from 1698444 to 2429379 tid 2 2023-06-27T12:44:47.547 INFO:tasks.rados.rados.0.smithi138.stdout:1589: write (excl) oid 9 current snap is 189 2023-06-27T12:44:47.548 INFO:tasks.rados.rados.0.smithi138.stdout:1589: seq_num 762 ranges {406934=575038,1640573=474593,2642791=164143} 2023-06-27T12:44:47.556 INFO:tasks.rados.rados.0.smithi138.stdout:1589: writing smithi138138488-9 from 406934 to 981972 tid 1 2023-06-27T12:44:47.564 INFO:tasks.rados.rados.0.smithi138.stdout:1589: writing smithi138138488-9 from 1640573 to 2115166 tid 2 2023-06-27T12:44:47.569 INFO:tasks.rados.rados.0.smithi138.stdout:1589: writing smithi138138488-9 from 2642791 to 2806934 tid 3 2023-06-27T12:44:47.570 INFO:tasks.rados.rados.0.smithi138.stdout:1590: read oid 6 snap -1 2023-06-27T12:44:47.570 INFO:tasks.rados.rados.0.smithi138.stdout:1590: expect (ObjNum 748 snap 185 seq_num 748) 2023-06-27T12:44:47.570 INFO:tasks.rados.rados.0.smithi138.stdout:1591: write oid 40 current snap is 189 2023-06-27T12:44:47.570 INFO:tasks.rados.rados.0.smithi138.stdout:1591: seq_num 763 ranges {423741=650287,1699055=618251} 2023-06-27T12:44:47.578 INFO:tasks.rados.rados.0.smithi138.stdout:1591: writing smithi138138488-40 from 423741 to 1074028 tid 1 2023-06-27T12:44:47.588 INFO:tasks.rados.rados.0.smithi138.stdout:1591: writing smithi138138488-40 from 1699055 to 2317306 tid 2 2023-06-27T12:44:47.589 INFO:tasks.rados.rados.0.smithi138.stdout:1592: write oid 45 current snap is 189 2023-06-27T12:44:47.589 INFO:tasks.rados.rados.0.smithi138.stdout:1588: finishing write tid 1 to smithi138138488-17 2023-06-27T12:44:47.589 INFO:tasks.rados.rados.0.smithi138.stdout:1592: seq_num 764 ranges {440548=725536,1757537=478262,2813990=26558} 2023-06-27T12:44:47.598 INFO:tasks.rados.rados.0.smithi138.stdout:1592: writing smithi138138488-45 from 440548 to 1166084 tid 1 2023-06-27T12:44:47.606 INFO:tasks.rados.rados.0.smithi138.stdout:1592: writing smithi138138488-45 from 1757537 to 2235799 tid 2 2023-06-27T12:44:47.610 INFO:tasks.rados.rados.0.smithi138.stdout:1592: writing smithi138138488-45 from 2813990 to 2840548 tid 3 2023-06-27T12:44:47.610 INFO:tasks.rados.rados.0.smithi138.stdout:1593: delete oid 18 current snap is 189 2023-06-27T12:44:47.611 INFO:tasks.rados.rados.0.smithi138.stdout:1590: expect (ObjNum 748 snap 185 seq_num 748) 2023-06-27T12:44:47.707 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:47.711 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:47.711 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:47.733 INFO:tasks.rados.rados.0.smithi138.stdout:1588: finishing write tid 2 to smithi138138488-17 2023-06-27T12:44:47.733 INFO:tasks.rados.rados.0.smithi138.stdout:1588: finishing write tid 3 to smithi138138488-17 2023-06-27T12:44:47.733 INFO:tasks.rados.rados.0.smithi138.stdout:1591: finishing write tid 1 to smithi138138488-40 2023-06-27T12:44:47.734 INFO:tasks.rados.rados.0.smithi138.stdout:1591: finishing write tid 2 to smithi138138488-40 2023-06-27T12:44:47.734 INFO:tasks.rados.rados.0.smithi138.stdout:1591: finishing write tid 3 to smithi138138488-40 2023-06-27T12:44:47.734 INFO:tasks.rados.rados.0.smithi138.stdout:1588: finishing write tid 4 to smithi138138488-17 2023-06-27T12:44:47.734 INFO:tasks.rados.rados.0.smithi138.stdout:1588: oid 17 updating version 0 to 273 2023-06-27T12:44:47.734 INFO:tasks.rados.rados.0.smithi138.stdout:1588: oid 17 version 273 is already newer than 273 2023-06-27T12:44:47.734 INFO:tasks.rados.rados.0.smithi138.stdout:1588: oid 17 version 273 is already newer than 273 2023-06-27T12:44:47.734 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 17 v 273 (ObjNum 761 snap 189 seq_num 761) dirty exists 2023-06-27T12:44:47.734 INFO:tasks.rados.rados.0.smithi138.stdout:1588: left oid 17 (ObjNum 761 snap 189 seq_num 761) 2023-06-27T12:44:47.735 INFO:tasks.rados.rados.0.smithi138.stdout:1592: finishing write tid 1 to smithi138138488-45 2023-06-27T12:44:47.735 INFO:tasks.rados.rados.0.smithi138.stdout:1591: finishing write tid 4 to smithi138138488-40 2023-06-27T12:44:47.735 INFO:tasks.rados.rados.0.smithi138.stdout:1591: oid 40 updating version 0 to 225 2023-06-27T12:44:47.735 INFO:tasks.rados.rados.0.smithi138.stdout:1591: oid 40 version 225 is already newer than 225 2023-06-27T12:44:47.735 INFO:tasks.rados.rados.0.smithi138.stdout:1591: oid 40 version 225 is already newer than 224 2023-06-27T12:44:47.735 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 40 v 225 (ObjNum 763 snap 189 seq_num 763) dirty exists 2023-06-27T12:44:47.735 INFO:tasks.rados.rados.0.smithi138.stdout:1591: left oid 40 (ObjNum 763 snap 189 seq_num 763) 2023-06-27T12:44:47.736 INFO:tasks.rados.rados.0.smithi138.stdout:1592: finishing write tid 2 to smithi138138488-45 2023-06-27T12:44:47.736 INFO:tasks.rados.rados.0.smithi138.stdout:1592: finishing write tid 3 to smithi138138488-45 2023-06-27T12:44:47.736 INFO:tasks.rados.rados.0.smithi138.stdout:1589: finishing write tid 1 to smithi138138488-9 2023-06-27T12:44:47.736 INFO:tasks.rados.rados.0.smithi138.stdout:1588: done (5 left) 2023-06-27T12:44:47.737 INFO:tasks.rados.rados.0.smithi138.stdout:1590: done (4 left) 2023-06-27T12:44:47.737 INFO:tasks.rados.rados.0.smithi138.stdout:1591: done (3 left) 2023-06-27T12:44:47.737 INFO:tasks.rados.rados.0.smithi138.stdout:1593: done (2 left) 2023-06-27T12:44:47.737 INFO:tasks.rados.rados.0.smithi138.stdout:1594: write (excl) oid 33 current snap is 189 2023-06-27T12:44:47.737 INFO:tasks.rados.rados.0.smithi138.stdout:1594: seq_num 766 ranges {474162=476034,1358148=481931,2352483=465303} 2023-06-27T12:44:47.741 INFO:tasks.rados.rados.0.smithi138.stdout:1594: writing smithi138138488-33 from 474162 to 950196 tid 1 2023-06-27T12:44:47.748 INFO:tasks.rados.rados.0.smithi138.stdout:1594: writing smithi138138488-33 from 1358148 to 1840079 tid 2 2023-06-27T12:44:47.757 INFO:tasks.rados.rados.0.smithi138.stdout:1594: writing smithi138138488-33 from 2352483 to 2817786 tid 3 2023-06-27T12:44:47.757 INFO:tasks.rados.rados.0.smithi138.stdout:1595: read oid 23 snap -1 2023-06-27T12:44:47.757 INFO:tasks.rados.rados.0.smithi138.stdout:1595: expect (ObjNum 716 snap 178 seq_num 716) 2023-06-27T12:44:47.758 INFO:tasks.rados.rados.0.smithi138.stdout:1596: read oid 4 snap -1 2023-06-27T12:44:47.758 INFO:tasks.rados.rados.0.smithi138.stdout:1596: expect (ObjNum 682 snap 167 seq_num 682) 2023-06-27T12:44:47.758 INFO:tasks.rados.rados.0.smithi138.stdout:1597: write oid 2 current snap is 189 2023-06-27T12:44:47.758 INFO:tasks.rados.rados.0.smithi138.stdout:1592: finishing write tid 4 to smithi138138488-45 2023-06-27T12:44:47.759 INFO:tasks.rados.rados.0.smithi138.stdout:1589: finishing write tid 2 to smithi138138488-9 2023-06-27T12:44:47.759 INFO:tasks.rados.rados.0.smithi138.stdout:1592: finishing write tid 5 to smithi138138488-45 2023-06-27T12:44:47.759 INFO:tasks.rados.rados.0.smithi138.stdout:1592: oid 45 updating version 0 to 489 2023-06-27T12:44:47.759 INFO:tasks.rados.rados.0.smithi138.stdout:1592: oid 45 version 489 is already newer than 489 2023-06-27T12:44:47.759 INFO:tasks.rados.rados.0.smithi138.stdout:1592: oid 45 version 489 is already newer than 489 2023-06-27T12:44:47.760 INFO:tasks.rados.rados.0.smithi138.stdout:1592: oid 45 version 489 is already newer than 489 2023-06-27T12:44:47.760 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 45 v 489 (ObjNum 764 snap 189 seq_num 764) dirty exists 2023-06-27T12:44:47.760 INFO:tasks.rados.rados.0.smithi138.stdout:1592: left oid 45 (ObjNum 764 snap 189 seq_num 764) 2023-06-27T12:44:47.760 INFO:tasks.rados.rados.0.smithi138.stdout:1589: finishing write tid 3 to smithi138138488-9 2023-06-27T12:44:47.760 INFO:tasks.rados.rados.0.smithi138.stdout:1589: finishing write tid 4 to smithi138138488-9 2023-06-27T12:44:47.761 INFO:tasks.rados.rados.0.smithi138.stdout:1597: seq_num 767 ranges {490969=551283,1816630=625589} 2023-06-27T12:44:47.764 INFO:tasks.rados.rados.0.smithi138.stdout:1597: writing smithi138138488-2 from 490969 to 1042252 tid 1 2023-06-27T12:44:47.775 INFO:tasks.rados.rados.0.smithi138.stdout:1597: writing smithi138138488-2 from 1816630 to 2442219 tid 2 2023-06-27T12:44:47.775 INFO:tasks.rados.rados.0.smithi138.stdout:1592: done (5 left) 2023-06-27T12:44:47.776 INFO:tasks.rados.rados.0.smithi138.stdout:1598: write (excl) oid 6 current snap is 189 2023-06-27T12:44:47.776 INFO:tasks.rados.rados.0.smithi138.stdout:1598: seq_num 768 ranges {507776=742885,1991465=485600} 2023-06-27T12:44:47.785 INFO:tasks.rados.rados.0.smithi138.stdout:1598: writing smithi138138488-6 from 507776 to 1250661 tid 1 2023-06-27T12:44:47.795 INFO:tasks.rados.rados.0.smithi138.stdout:1598: writing smithi138138488-6 from 1991465 to 2477065 tid 2 2023-06-27T12:44:47.795 INFO:tasks.rados.rados.0.smithi138.stdout:1599: read oid 29 snap -1 2023-06-27T12:44:47.795 INFO:tasks.rados.rados.0.smithi138.stdout:1599: expect deleted 2023-06-27T12:44:47.795 INFO:tasks.rados.rados.0.smithi138.stdout:1600: write (excl) oid 39 current snap is 189 2023-06-27T12:44:47.795 INFO:tasks.rados.rados.0.smithi138.stdout:1589: finishing write tid 5 to smithi138138488-9 2023-06-27T12:44:47.795 INFO:tasks.rados.rados.0.smithi138.stdout:1589: oid 9 updating version 0 to 455 2023-06-27T12:44:47.796 INFO:tasks.rados.rados.0.smithi138.stdout:1589: oid 9 version 455 is already newer than 455 2023-06-27T12:44:47.796 INFO:tasks.rados.rados.0.smithi138.stdout:1589: oid 9 version 455 is already newer than 452 2023-06-27T12:44:47.796 INFO:tasks.rados.rados.0.smithi138.stdout:1589: oid 9 version 455 is already newer than 455 2023-06-27T12:44:47.796 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 455 (ObjNum 762 snap 189 seq_num 762) dirty exists 2023-06-27T12:44:47.796 INFO:tasks.rados.rados.0.smithi138.stdout:1589: left oid 9 (ObjNum 762 snap 189 seq_num 762) 2023-06-27T12:44:47.796 INFO:tasks.rados.rados.0.smithi138.stdout:1600: seq_num 769 ranges {524583=418134,1649947=629258,2834752=89831} 2023-06-27T12:44:47.802 INFO:tasks.rados.rados.0.smithi138.stdout:1600: writing smithi138138488-39 from 524583 to 942717 tid 1 2023-06-27T12:44:47.812 INFO:tasks.rados.rados.0.smithi138.stdout:1600: writing smithi138138488-39 from 1649947 to 2279205 tid 2 2023-06-27T12:44:47.816 INFO:tasks.rados.rados.0.smithi138.stdout:1600: writing smithi138138488-39 from 2834752 to 2924583 tid 3 2023-06-27T12:44:47.817 INFO:tasks.rados.rados.0.smithi138.stdout:1589: done (7 left) 2023-06-27T12:44:47.817 INFO:tasks.rados.rados.0.smithi138.stdout:1601: delete oid 21 current snap is 189 2023-06-27T12:44:47.817 INFO:tasks.rados.rados.0.smithi138.stdout:1595: expect (ObjNum 716 snap 178 seq_num 716) 2023-06-27T12:44:47.908 INFO:tasks.rados.rados.0.smithi138.stdout:1594: finishing write tid 1 to smithi138138488-33 2023-06-27T12:44:47.908 INFO:tasks.rados.rados.0.smithi138.stdout:1596: expect (ObjNum 682 snap 167 seq_num 682) 2023-06-27T12:44:48.139 INFO:tasks.rados.rados.0.smithi138.stdout:1594: finishing write tid 2 to smithi138138488-33 2023-06-27T12:44:48.139 INFO:tasks.rados.rados.0.smithi138.stdout:1594: finishing write tid 3 to smithi138138488-33 2023-06-27T12:44:48.139 INFO:tasks.rados.rados.0.smithi138.stdout:1594: finishing write tid 4 to smithi138138488-33 2023-06-27T12:44:48.139 INFO:tasks.rados.rados.0.smithi138.stdout:1594: finishing write tid 5 to smithi138138488-33 2023-06-27T12:44:48.140 INFO:tasks.rados.rados.0.smithi138.stdout:1594: oid 33 updating version 0 to 230 2023-06-27T12:44:48.140 INFO:tasks.rados.rados.0.smithi138.stdout:1594: oid 33 version 230 is already newer than 230 2023-06-27T12:44:48.140 INFO:tasks.rados.rados.0.smithi138.stdout:1594: oid 33 version 230 is already newer than 230 2023-06-27T12:44:48.140 INFO:tasks.rados.rados.0.smithi138.stdout:1594: oid 33 version 230 is already newer than 226 2023-06-27T12:44:48.140 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 33 v 230 (ObjNum 766 snap 189 seq_num 766) dirty exists 2023-06-27T12:44:48.140 INFO:tasks.rados.rados.0.smithi138.stdout:1594: left oid 33 (ObjNum 766 snap 189 seq_num 766) 2023-06-27T12:44:48.140 INFO:tasks.rados.rados.0.smithi138.stdout:1597: finishing write tid 1 to smithi138138488-2 2023-06-27T12:44:48.141 INFO:tasks.rados.rados.0.smithi138.stdout:1600: finishing write tid 1 to smithi138138488-39 2023-06-27T12:44:48.141 INFO:tasks.rados.rados.0.smithi138.stdout:1594: done (7 left) 2023-06-27T12:44:48.141 INFO:tasks.rados.rados.0.smithi138.stdout:1595: done (6 left) 2023-06-27T12:44:48.141 INFO:tasks.rados.rados.0.smithi138.stdout:1596: done (5 left) 2023-06-27T12:44:48.141 INFO:tasks.rados.rados.0.smithi138.stdout:1599: done (4 left) 2023-06-27T12:44:48.141 INFO:tasks.rados.rados.0.smithi138.stdout:1601: done (3 left) 2023-06-27T12:44:48.142 INFO:tasks.rados.rados.0.smithi138.stdout:1602: write oid 37 current snap is 189 2023-06-27T12:44:48.142 INFO:tasks.rados.rados.0.smithi138.stdout:1597: finishing write tid 2 to smithi138138488-2 2023-06-27T12:44:48.142 INFO:tasks.rados.rados.0.smithi138.stdout:1597: finishing write tid 3 to smithi138138488-2 2023-06-27T12:44:48.142 INFO:tasks.rados.rados.0.smithi138.stdout:1597: finishing write tid 4 to smithi138138488-2 2023-06-27T12:44:48.142 INFO:tasks.rados.rados.0.smithi138.stdout:1597: oid 2 updating version 0 to 493 2023-06-27T12:44:48.142 INFO:tasks.rados.rados.0.smithi138.stdout:1597: oid 2 version 493 is already newer than 493 2023-06-27T12:44:48.142 INFO:tasks.rados.rados.0.smithi138.stdout:1597: oid 2 version 493 is already newer than 493 2023-06-27T12:44:48.142 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 2 v 493 (ObjNum 767 snap 189 seq_num 767) dirty exists 2023-06-27T12:44:48.143 INFO:tasks.rados.rados.0.smithi138.stdout:1597: left oid 2 (ObjNum 767 snap 189 seq_num 767) 2023-06-27T12:44:48.143 INFO:tasks.rados.rados.0.smithi138.stdout:1598: finishing write tid 1 to smithi138138488-6 2023-06-27T12:44:48.143 INFO:tasks.rados.rados.0.smithi138.stdout:1600: finishing write tid 2 to smithi138138488-39 2023-06-27T12:44:48.143 INFO:tasks.rados.rados.0.smithi138.stdout:1598: finishing write tid 2 to smithi138138488-6 2023-06-27T12:44:48.143 INFO:tasks.rados.rados.0.smithi138.stdout:1598: finishing write tid 3 to smithi138138488-6 2023-06-27T12:44:48.143 INFO:tasks.rados.rados.0.smithi138.stdout:1598: finishing write tid 4 to smithi138138488-6 2023-06-27T12:44:48.143 INFO:tasks.rados.rados.0.smithi138.stdout:1598: oid 6 updating version 0 to 220 2023-06-27T12:44:48.143 INFO:tasks.rados.rados.0.smithi138.stdout:1598: oid 6 version 220 is already newer than 220 2023-06-27T12:44:48.144 INFO:tasks.rados.rados.0.smithi138.stdout:1598: oid 6 version 220 is already newer than 217 2023-06-27T12:44:48.144 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 220 (ObjNum 768 snap 189 seq_num 768) dirty exists 2023-06-27T12:44:48.144 INFO:tasks.rados.rados.0.smithi138.stdout:1598: left oid 6 (ObjNum 768 snap 189 seq_num 768) 2023-06-27T12:44:48.144 INFO:tasks.rados.rados.0.smithi138.stdout:1600: finishing write tid 3 to smithi138138488-39 2023-06-27T12:44:48.144 INFO:tasks.rados.rados.0.smithi138.stdout:1600: finishing write tid 4 to smithi138138488-39 2023-06-27T12:44:48.144 INFO:tasks.rados.rados.0.smithi138.stdout:1600: finishing write tid 5 to smithi138138488-39 2023-06-27T12:44:48.145 INFO:tasks.rados.rados.0.smithi138.stdout:1600: oid 39 updating version 0 to 286 2023-06-27T12:44:48.145 INFO:tasks.rados.rados.0.smithi138.stdout:1600: oid 39 version 286 is already newer than 282 2023-06-27T12:44:48.145 INFO:tasks.rados.rados.0.smithi138.stdout:1600: oid 39 version 286 is already newer than 286 2023-06-27T12:44:48.145 INFO:tasks.rados.rados.0.smithi138.stdout:1600: oid 39 version 286 is already newer than 286 2023-06-27T12:44:48.145 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 39 v 286 (ObjNum 769 snap 189 seq_num 769) dirty exists 2023-06-27T12:44:48.145 INFO:tasks.rados.rados.0.smithi138.stdout:1600: left oid 39 (ObjNum 769 snap 189 seq_num 769) 2023-06-27T12:44:48.145 INFO:tasks.rados.rados.0.smithi138.stdout:1602: seq_num 771 ranges {558197=568632,1650558=632927,2773245=184952} 2023-06-27T12:44:48.146 INFO:tasks.rados.rados.0.smithi138.stdout:1602: writing smithi138138488-37 from 558197 to 1126829 tid 1 2023-06-27T12:44:48.153 INFO:tasks.rados.rados.0.smithi138.stdout:1602: writing smithi138138488-37 from 1650558 to 2283485 tid 2 2023-06-27T12:44:48.158 INFO:tasks.rados.rados.0.smithi138.stdout:1602: writing smithi138138488-37 from 2773245 to 2958197 tid 3 2023-06-27T12:44:48.158 INFO:tasks.rados.rados.0.smithi138.stdout:1597: done (3 left) 2023-06-27T12:44:48.159 INFO:tasks.rados.rados.0.smithi138.stdout:1598: done (2 left) 2023-06-27T12:44:48.159 INFO:tasks.rados.rados.0.smithi138.stdout:1600: done (1 left) 2023-06-27T12:44:48.159 INFO:tasks.rados.rados.0.smithi138.stdout:1603: read oid 39 snap -1 2023-06-27T12:44:48.159 INFO:tasks.rados.rados.0.smithi138.stdout:1603: expect (ObjNum 769 snap 189 seq_num 769) 2023-06-27T12:44:48.159 INFO:tasks.rados.rados.0.smithi138.stdout:1604: read oid 42 snap -1 2023-06-27T12:44:48.159 INFO:tasks.rados.rados.0.smithi138.stdout:1604: expect (ObjNum 752 snap 185 seq_num 752) 2023-06-27T12:44:48.159 INFO:tasks.rados.rados.0.smithi138.stdout:1605: write (excl) oid 7 current snap is 189 2023-06-27T12:44:48.160 INFO:tasks.rados.rados.0.smithi138.stdout:1605: seq_num 772 ranges {575004=643881,1709040=492938,2917021=57983} 2023-06-27T12:44:48.168 INFO:tasks.rados.rados.0.smithi138.stdout:1605: writing smithi138138488-7 from 575004 to 1218885 tid 1 2023-06-27T12:44:48.176 INFO:tasks.rados.rados.0.smithi138.stdout:1605: writing smithi138138488-7 from 1709040 to 2201978 tid 2 2023-06-27T12:44:48.181 INFO:tasks.rados.rados.0.smithi138.stdout:1605: writing smithi138138488-7 from 2917021 to 2975004 tid 3 2023-06-27T12:44:48.181 INFO:tasks.rados.rados.0.smithi138.stdout:1606: snap_remove snap 177 2023-06-27T12:44:48.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~253s 2023-06-27T12:44:48.507 INFO:tasks.rados.rados.0.smithi138.stdout:1606: done (4 left) 2023-06-27T12:44:48.507 INFO:tasks.rados.rados.0.smithi138.stdout:1607: read oid 29 snap -1 2023-06-27T12:44:48.507 INFO:tasks.rados.rados.0.smithi138.stdout:1607: expect deleted 2023-06-27T12:44:48.507 INFO:tasks.rados.rados.0.smithi138.stdout:1608: snap_create 2023-06-27T12:44:48.508 INFO:tasks.rados.rados.0.smithi138.stdout:1602: finishing write tid 1 to smithi138138488-37 2023-06-27T12:44:48.508 INFO:tasks.rados.rados.0.smithi138.stdout:1604: expect (ObjNum 752 snap 185 seq_num 752) 2023-06-27T12:44:48.655 INFO:tasks.rados.rados.0.smithi138.stdout:1602: finishing write tid 2 to smithi138138488-37 2023-06-27T12:44:48.656 INFO:tasks.rados.rados.0.smithi138.stdout:1603: expect (ObjNum 769 snap 189 seq_num 769) 2023-06-27T12:44:48.841 INFO:tasks.rados.rados.0.smithi138.stdout:1605: finishing write tid 1 to smithi138138488-7 2023-06-27T12:44:48.842 INFO:tasks.rados.rados.0.smithi138.stdout:1602: finishing write tid 3 to smithi138138488-37 2023-06-27T12:44:48.842 INFO:tasks.rados.rados.0.smithi138.stdout:1602: finishing write tid 4 to smithi138138488-37 2023-06-27T12:44:48.842 INFO:tasks.rados.rados.0.smithi138.stdout:1602: finishing write tid 5 to smithi138138488-37 2023-06-27T12:44:48.842 INFO:tasks.rados.rados.0.smithi138.stdout:1602: oid 37 updating version 0 to 327 2023-06-27T12:44:48.842 INFO:tasks.rados.rados.0.smithi138.stdout:1602: oid 37 version 327 is already newer than 327 2023-06-27T12:44:48.842 INFO:tasks.rados.rados.0.smithi138.stdout:1602: oid 37 version 327 is already newer than 327 2023-06-27T12:44:48.843 INFO:tasks.rados.rados.0.smithi138.stdout:1602: oid 37 version 327 is already newer than 325 2023-06-27T12:44:48.843 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 327 (ObjNum 771 snap 189 seq_num 771) dirty exists 2023-06-27T12:44:48.843 INFO:tasks.rados.rados.0.smithi138.stdout:1602: left oid 37 (ObjNum 771 snap 189 seq_num 771) 2023-06-27T12:44:48.843 INFO:tasks.rados.rados.0.smithi138.stdout:1605: finishing write tid 2 to smithi138138488-7 2023-06-27T12:44:48.843 INFO:tasks.rados.rados.0.smithi138.stdout:1605: finishing write tid 3 to smithi138138488-7 2023-06-27T12:44:48.843 INFO:tasks.rados.rados.0.smithi138.stdout:1605: finishing write tid 4 to smithi138138488-7 2023-06-27T12:44:48.843 INFO:tasks.rados.rados.0.smithi138.stdout:1605: finishing write tid 5 to smithi138138488-7 2023-06-27T12:44:48.843 INFO:tasks.rados.rados.0.smithi138.stdout:1605: oid 7 updating version 0 to 261 2023-06-27T12:44:48.844 INFO:tasks.rados.rados.0.smithi138.stdout:1605: oid 7 updating version 261 to 265 2023-06-27T12:44:48.844 INFO:tasks.rados.rados.0.smithi138.stdout:1605: oid 7 version 265 is already newer than 265 2023-06-27T12:44:48.844 INFO:tasks.rados.rados.0.smithi138.stdout:1605: oid 7 version 265 is already newer than 265 2023-06-27T12:44:48.844 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 265 (ObjNum 772 snap 189 seq_num 772) dirty exists 2023-06-27T12:44:48.844 INFO:tasks.rados.rados.0.smithi138.stdout:1605: left oid 7 (ObjNum 772 snap 189 seq_num 772) 2023-06-27T12:44:49.510 INFO:tasks.rados.rados.0.smithi138.stdout:1602: done (5 left) 2023-06-27T12:44:49.511 INFO:tasks.rados.rados.0.smithi138.stdout:1603: done (4 left) 2023-06-27T12:44:49.511 INFO:tasks.rados.rados.0.smithi138.stdout:1604: done (3 left) 2023-06-27T12:44:49.511 INFO:tasks.rados.rados.0.smithi138.stdout:1605: done (2 left) 2023-06-27T12:44:49.511 INFO:tasks.rados.rados.0.smithi138.stdout:1607: done (1 left) 2023-06-27T12:44:49.511 INFO:tasks.rados.rados.0.smithi138.stdout:1608: done (0 left) 2023-06-27T12:44:49.512 INFO:tasks.rados.rados.0.smithi138.stdout:1609: read oid 8 snap -1 2023-06-27T12:44:49.512 INFO:tasks.rados.rados.0.smithi138.stdout:1609: expect (ObjNum 664 snap 164 seq_num 664) 2023-06-27T12:44:49.512 INFO:tasks.rados.rados.0.smithi138.stdout:1610: read oid 38 snap 182 2023-06-27T12:44:49.512 INFO:tasks.rados.rados.0.smithi138.stdout:1610: expect (ObjNum 698 snap 171 seq_num 698) 2023-06-27T12:44:49.512 INFO:tasks.rados.rados.0.smithi138.stdout:1611: write (excl) oid 48 current snap is 190 2023-06-27T12:44:49.512 INFO:tasks.rados.rados.0.smithi138.stdout:1611: seq_num 773 ranges {591811=719130,1767522=636596,2828091=163720} 2023-06-27T12:44:49.521 INFO:tasks.rados.rados.0.smithi138.stdout:1611: writing smithi138138488-48 from 591811 to 1310941 tid 1 2023-06-27T12:44:49.529 INFO:tasks.rados.rados.0.smithi138.stdout:1611: writing smithi138138488-48 from 1767522 to 2404118 tid 2 2023-06-27T12:44:49.533 INFO:tasks.rados.rados.0.smithi138.stdout:1611: writing smithi138138488-48 from 2828091 to 2991811 tid 3 2023-06-27T12:44:49.534 INFO:tasks.rados.rados.0.smithi138.stdout:1612: write oid 19 current snap is 190 2023-06-27T12:44:49.534 INFO:tasks.rados.rados.0.smithi138.stdout:1612: seq_num 774 ranges {608618=794379,2109651=780254} 2023-06-27T12:44:49.544 INFO:tasks.rados.rados.0.smithi138.stdout:1612: writing smithi138138488-19 from 608618 to 1402997 tid 1 2023-06-27T12:44:49.556 INFO:tasks.rados.rados.0.smithi138.stdout:1612: writing smithi138138488-19 from 2109651 to 2889905 tid 2 2023-06-27T12:44:49.557 INFO:tasks.rados.rados.0.smithi138.stdout:1613: snap_create 2023-06-27T12:44:49.600 INFO:tasks.rados.rados.0.smithi138.stdout:1609: expect (ObjNum 664 snap 164 seq_num 664) 2023-06-27T12:44:49.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:49.797 INFO:tasks.rados.rados.0.smithi138.stdout:1610: expect (ObjNum 698 snap 171 seq_num 698) 2023-06-27T12:44:49.840 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:49.835+0000 7f729918c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:49.843 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:49.839+0000 7f729918c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:49.887 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:49.890 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:49.891 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:49.939 INFO:tasks.rados.rados.0.smithi138.stdout:1611: finishing write tid 1 to smithi138138488-48 2023-06-27T12:44:49.940 INFO:tasks.rados.rados.0.smithi138.stdout:1612: finishing write tid 1 to smithi138138488-19 2023-06-27T12:44:49.940 INFO:tasks.rados.rados.0.smithi138.stdout:1612: finishing write tid 2 to smithi138138488-19 2023-06-27T12:44:49.940 INFO:tasks.rados.rados.0.smithi138.stdout:1612: finishing write tid 3 to smithi138138488-19 2023-06-27T12:44:49.940 INFO:tasks.rados.rados.0.smithi138.stdout:1612: finishing write tid 4 to smithi138138488-19 2023-06-27T12:44:49.940 INFO:tasks.rados.rados.0.smithi138.stdout:1612: oid 19 updating version 0 to 465 2023-06-27T12:44:49.940 INFO:tasks.rados.rados.0.smithi138.stdout:1612: oid 19 version 465 is already newer than 465 2023-06-27T12:44:49.941 INFO:tasks.rados.rados.0.smithi138.stdout:1612: oid 19 version 465 is already newer than 465 2023-06-27T12:44:49.941 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 465 (ObjNum 774 snap 190 seq_num 774) dirty exists 2023-06-27T12:44:49.941 INFO:tasks.rados.rados.0.smithi138.stdout:1612: left oid 19 (ObjNum 774 snap 190 seq_num 774) 2023-06-27T12:44:49.941 INFO:tasks.rados.rados.0.smithi138.stdout:1611: finishing write tid 2 to smithi138138488-48 2023-06-27T12:44:49.941 INFO:tasks.rados.rados.0.smithi138.stdout:1611: finishing write tid 3 to smithi138138488-48 2023-06-27T12:44:49.941 INFO:tasks.rados.rados.0.smithi138.stdout:1611: finishing write tid 4 to smithi138138488-48 2023-06-27T12:44:49.941 INFO:tasks.rados.rados.0.smithi138.stdout:1611: finishing write tid 5 to smithi138138488-48 2023-06-27T12:44:49.941 INFO:tasks.rados.rados.0.smithi138.stdout:1611: oid 48 updating version 0 to 237 2023-06-27T12:44:49.942 INFO:tasks.rados.rados.0.smithi138.stdout:1611: oid 48 updating version 237 to 241 2023-06-27T12:44:49.942 INFO:tasks.rados.rados.0.smithi138.stdout:1611: oid 48 version 241 is already newer than 241 2023-06-27T12:44:49.942 INFO:tasks.rados.rados.0.smithi138.stdout:1611: oid 48 version 241 is already newer than 241 2023-06-27T12:44:49.942 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 48 v 241 (ObjNum 773 snap 190 seq_num 773) dirty exists 2023-06-27T12:44:49.942 INFO:tasks.rados.rados.0.smithi138.stdout:1611: left oid 48 (ObjNum 773 snap 190 seq_num 773) 2023-06-27T12:44:50.467 INFO:tasks.rados.rados.0.smithi138.stdout:1609: done (4 left) 2023-06-27T12:44:50.468 INFO:tasks.rados.rados.0.smithi138.stdout:1610: done (3 left) 2023-06-27T12:44:50.468 INFO:tasks.rados.rados.0.smithi138.stdout:1611: done (2 left) 2023-06-27T12:44:50.468 INFO:tasks.rados.rados.0.smithi138.stdout:1612: done (1 left) 2023-06-27T12:44:50.468 INFO:tasks.rados.rados.0.smithi138.stdout:1613: done (0 left) 2023-06-27T12:44:50.468 INFO:tasks.rados.rados.0.smithi138.stdout:1614: delete oid 18 current snap is 191 2023-06-27T12:44:50.490 INFO:tasks.rados.rados.0.smithi138.stdout:1614: done (0 left) 2023-06-27T12:44:50.491 INFO:tasks.rados.rados.0.smithi138.stdout:1615: read oid 8 snap -1 2023-06-27T12:44:50.491 INFO:tasks.rados.rados.0.smithi138.stdout:1615: expect (ObjNum 664 snap 164 seq_num 664) 2023-06-27T12:44:50.491 INFO:tasks.rados.rados.0.smithi138.stdout:1616: read oid 24 snap -1 2023-06-27T12:44:50.491 INFO:tasks.rados.rados.0.smithi138.stdout:1616: expect deleted 2023-06-27T12:44:50.491 INFO:tasks.rados.rados.0.smithi138.stdout:1617: delete oid 28 current snap is 191 2023-06-27T12:44:50.520 INFO:tasks.rados.rados.0.smithi138.stdout:1616: done (2 left) 2023-06-27T12:44:50.521 INFO:tasks.rados.rados.0.smithi138.stdout:1617: done (1 left) 2023-06-27T12:44:50.521 INFO:tasks.rados.rados.0.smithi138.stdout:1618: delete oid 47 current snap is 191 2023-06-27T12:44:50.530 INFO:tasks.rados.rados.0.smithi138.stdout:1615: expect (ObjNum 664 snap 164 seq_num 664) 2023-06-27T12:44:50.713 INFO:tasks.rados.rados.0.smithi138.stdout:1615: done (1 left) 2023-06-27T12:44:50.714 INFO:tasks.rados.rados.0.smithi138.stdout:1618: done (0 left) 2023-06-27T12:44:50.714 INFO:tasks.rados.rados.0.smithi138.stdout:1619: write oid 27 current snap is 191 2023-06-27T12:44:50.714 INFO:tasks.rados.rados.0.smithi138.stdout:1619: seq_num 778 ranges {675846=411728,1543579=787592,2848853=226993} 2023-06-27T12:44:50.719 INFO:tasks.rados.rados.0.smithi138.stdout:1619: writing smithi138138488-27 from 675846 to 1087574 tid 1 2023-06-27T12:44:50.727 INFO:tasks.rados.rados.0.smithi138.stdout:1619: writing smithi138138488-27 from 1543579 to 2331171 tid 2 2023-06-27T12:44:50.732 INFO:tasks.rados.rados.0.smithi138.stdout:1619: writing smithi138138488-27 from 2848853 to 3075846 tid 3 2023-06-27T12:44:50.732 INFO:tasks.rados.rados.0.smithi138.stdout:1620: read oid 40 snap -1 2023-06-27T12:44:50.732 INFO:tasks.rados.rados.0.smithi138.stdout:1620: expect (ObjNum 763 snap 189 seq_num 763) 2023-06-27T12:44:50.733 INFO:tasks.rados.rados.0.smithi138.stdout:1621: write (excl) oid 43 current snap is 191 2023-06-27T12:44:50.733 INFO:tasks.rados.rados.0.smithi138.stdout:1621: seq_num 779 ranges {692653=486977,1602061=647603,2992629=100024} 2023-06-27T12:44:50.740 INFO:tasks.rados.rados.0.smithi138.stdout:1621: writing smithi138138488-43 from 692653 to 1179630 tid 1 2023-06-27T12:44:50.749 INFO:tasks.rados.rados.0.smithi138.stdout:1621: writing smithi138138488-43 from 1602061 to 2249664 tid 2 2023-06-27T12:44:50.755 INFO:tasks.rados.rados.0.smithi138.stdout:1621: writing smithi138138488-43 from 2992629 to 3092653 tid 3 2023-06-27T12:44:50.755 INFO:tasks.rados.rados.0.smithi138.stdout:1622: write oid 13 current snap is 191 2023-06-27T12:44:50.755 INFO:tasks.rados.rados.0.smithi138.stdout:1622: seq_num 780 ranges {709460=562226,2060543=791261} 2023-06-27T12:44:50.765 INFO:tasks.rados.rados.0.smithi138.stdout:1622: writing smithi138138488-13 from 709460 to 1271686 tid 1 2023-06-27T12:44:50.777 INFO:tasks.rados.rados.0.smithi138.stdout:1622: writing smithi138138488-13 from 2060543 to 2851804 tid 2 2023-06-27T12:44:50.777 INFO:tasks.rados.rados.0.smithi138.stdout:1623: delete oid 30 current snap is 191 2023-06-27T12:44:50.777 INFO:tasks.rados.rados.0.smithi138.stdout:1619: finishing write tid 1 to smithi138138488-27 2023-06-27T12:44:50.777 INFO:tasks.rados.rados.0.smithi138.stdout:1619: finishing write tid 2 to smithi138138488-27 2023-06-27T12:44:50.778 INFO:tasks.rados.rados.0.smithi138.stdout:1619: finishing write tid 3 to smithi138138488-27 2023-06-27T12:44:50.778 INFO:tasks.rados.rados.0.smithi138.stdout:1619: finishing write tid 4 to smithi138138488-27 2023-06-27T12:44:50.778 INFO:tasks.rados.rados.0.smithi138.stdout:1619: finishing write tid 5 to smithi138138488-27 2023-06-27T12:44:50.779 INFO:tasks.rados.rados.0.smithi138.stdout:1619: oid 27 updating version 0 to 285 2023-06-27T12:44:50.779 INFO:tasks.rados.rados.0.smithi138.stdout:1619: oid 27 updating version 285 to 287 2023-06-27T12:44:50.779 INFO:tasks.rados.rados.0.smithi138.stdout:1619: oid 27 version 287 is already newer than 287 2023-06-27T12:44:50.779 INFO:tasks.rados.rados.0.smithi138.stdout:1619: oid 27 version 287 is already newer than 287 2023-06-27T12:44:50.779 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 27 v 287 (ObjNum 778 snap 191 seq_num 778) dirty exists 2023-06-27T12:44:50.779 INFO:tasks.rados.rados.0.smithi138.stdout:1619: left oid 27 (ObjNum 778 snap 191 seq_num 778) 2023-06-27T12:44:50.785 INFO:tasks.rados.rados.0.smithi138.stdout:1619: done (4 left) 2023-06-27T12:44:50.786 INFO:tasks.rados.rados.0.smithi138.stdout:1623: done (3 left) 2023-06-27T12:44:50.786 INFO:tasks.rados.rados.0.smithi138.stdout:1624: snap_remove snap 186 2023-06-27T12:44:50.941 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:50.941 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:50.942 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:51.070 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:51.063+0000 7fbdae7fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:51.073 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:51.069+0000 7fbdae7fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:51.265 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:51.267 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:51.278 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2023-06-27T12:44:50.461959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":102511899,"num_objects":412,"num_object_clones":365,"num_object_copies":1236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":2358,"num_read_kb":1743590,"num_write":2164,"num_write_kb":629571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3977,"ondisk_log_size":3977,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1203000,"kb_used_data":1067032,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376284360,"statfs":{"total":386547056640,"available":385315184640,"internally_reserved":0,"allocated":1092640768,"data_stored":1264090654,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8285884,"num_objects":15,"num_object_clones":15,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":62,"num_read_kb":42562,"num_write":58,"num_write_kb":15635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.967942"},"pg_stats":[{"pgid":"3.f","version":"387'58","reported_seq":930,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:47.511640+0000","last_change":"2023-06-27T12:44:47.516920+0000","last_active":"2023-06-27T12:44:47.511640+0000","last_peered":"2023-06-27T12:44:47.511640+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:47.511640+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2655671,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":15,"num_read_kb":8,"num_write":37,"num_write_kb":10467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"393'273","reported_seq":1241,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:47.563255+0000","last_change":"2023-06-27T12:44:47.553417+0000","last_active":"2023-06-27T12:44:47.563255+0000","last_peered":"2023-06-27T12:44:47.563255+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:47.563255+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7546838,"num_objects":35,"num_object_clones":31,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":171,"num_read_kb":213040,"num_write":184,"num_write_kb":55882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":35}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"393'327","reported_seq":1228,"reported_epoch":394,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:48.513397+0000","last_change":"2023-06-27T12:44:48.566054+0000","last_active":"2023-06-27T12:44:48.513397+0000","last_peered":"2023-06-27T12:44:48.513397+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:48.513397+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11261251,"num_objects":37,"num_object_clones":32,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":276,"num_read_kb":186830,"num_write":205,"num_write_kb":59601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":37}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"393'230","reported_seq":1162,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:48.193645+0000","last_change":"2023-06-27T12:44:47.522664+0000","last_active":"2023-06-27T12:44:48.193645+0000","last_peered":"2023-06-27T12:44:48.193645+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:48.193645+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8336767,"num_objects":33,"num_object_clones":29,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":209,"num_read_kb":148866,"num_write":152,"num_write_kb":43433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":33}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":670,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:47.513290+0000","last_change":"2023-06-27T12:44:47.521899+0000","last_active":"2023-06-27T12:44:47.513290+0000","last_peered":"2023-06-27T12:44:47.513290+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:47.513290+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":76842,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"376'455","reported_seq":1234,"reported_epoch":393,"state":"active+clean","last_fresh":"2023-06-27T12:44:47.510147+0000","last_change":"2023-06-27T12:44:47.522338+0000","last_active":"2023-06-27T12:44:47.510147+0000","last_peered":"2023-06-27T12:44:47.510147+0000","last_clean":"2023-06-27T12:44:47.510147+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:47.510147+0000","last_undegraded":"2023-06-27T12:44:47.510147+0000","last_fullsized":"2023-06-27T12:44:47.510147+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10471388,"num_objects":49,"num_object_clones":43,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":333,"num_read_kb":211120,"num_write":265,"num_write_kb":75831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"393'265","reported_seq":1048,"reported_epoch":393,"state":"active+clean","last_fresh":"2023-06-27T12:44:48.263380+0000","last_change":"2023-06-27T12:44:47.601950+0000","last_active":"2023-06-27T12:44:48.263380+0000","last_peered":"2023-06-27T12:44:48.263380+0000","last_clean":"2023-06-27T12:44:48.263380+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:48.263380+0000","last_undegraded":"2023-06-27T12:44:48.263380+0000","last_fullsized":"2023-06-27T12:44:48.263380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5428991,"num_objects":31,"num_object_clones":28,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":143,"num_read_kb":85556,"num_write":152,"num_write_kb":43826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"390'83","reported_seq":975,"reported_epoch":394,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:48.511197+0000","last_change":"2023-06-27T12:44:48.548978+0000","last_active":"2023-06-27T12:44:48.511197+0000","last_peered":"2023-06-27T12:44:48.511197+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:48.511197+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2672478,"num_objects":10,"num_object_clones":9,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":47129,"num_write":50,"num_write_kb":15046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":662,"reported_epoch":393,"state":"active+undersized","last_fresh":"2023-06-27T12:44:47.515890+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:47.515890+0000","last_peered":"2023-06-27T12:44:47.515890+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:47.515890+0000","last_undegraded":"2023-06-27T12:44:47.515890+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":879,"reported_epoch":394,"state":"active+undersized","last_fresh":"2023-06-27T12:44:48.512030+0000","last_change":"2023-06-27T12:44:48.522117+0000","last_active":"2023-06-27T12:44:48.512030+0000","last_peered":"2023-06-27T12:44:48.512030+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:48.512030+0000","last_undegraded":"2023-06-27T12:44:48.512030+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":637,"reported_epoch":393,"state":"active+clean","last_fresh":"2023-06-27T12:44:47.510623+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:47.510623+0000","last_peered":"2023-06-27T12:44:47.510623+0000","last_clean":"2023-06-27T12:44:47.510623+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:47.510623+0000","last_undegraded":"2023-06-27T12:44:47.510623+0000","last_fullsized":"2023-06-27T12:44:47.510623+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"393'220","reported_seq":1125,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:47.881116+0000","last_change":"2023-06-27T12:44:47.570545+0000","last_active":"2023-06-27T12:44:47.881116+0000","last_peered":"2023-06-27T12:44:47.881116+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:47.881116+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7681294,"num_objects":25,"num_object_clones":20,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":94,"num_read_kb":78897,"num_write":140,"num_write_kb":38532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":632,"reported_epoch":394,"state":"active+undersized","last_fresh":"2023-06-27T12:44:48.515802+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:48.515802+0000","last_peered":"2023-06-27T12:44:48.515802+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:48.515802+0000","last_undegraded":"2023-06-27T12:44:48.515802+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"393'493","reported_seq":813,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:47.779722+0000","last_change":"2023-06-27T12:44:47.523716+0000","last_active":"2023-06-27T12:44:47.779722+0000","last_peered":"2023-06-27T12:44:47.779722+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:47.779722+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5731517,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":84,"num_read_kb":60343,"num_write":149,"num_write_kb":45929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":629,"reported_epoch":394,"state":"active+clean","last_fresh":"2023-06-27T12:44:48.513246+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:48.513246+0000","last_peered":"2023-06-27T12:44:48.513246+0000","last_clean":"2023-06-27T12:44:48.513246+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:48.513246+0000","last_undegraded":"2023-06-27T12:44:48.513246+0000","last_fullsized":"2023-06-27T12:44:48.513246+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":389,"reported_epoch":393,"state":"active+undersized","last_fresh":"2023-06-27T12:44:47.512120+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:47.512120+0000","last_peered":"2023-06-27T12:44:47.512120+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:47.512120+0000","last_undegraded":"2023-06-27T12:44:47.512120+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"393'457","reported_seq":1359,"reported_epoch":394,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:48.519632+0000","last_change":"2023-06-27T12:44:48.568733+0000","last_active":"2023-06-27T12:44:48.519632+0000","last_peered":"2023-06-27T12:44:48.519632+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:48.519632+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10034439,"num_objects":55,"num_object_clones":49,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":55,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":337,"num_read_kb":203163,"num_write":309,"num_write_kb":90223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":55}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":637,"reported_epoch":393,"state":"active+clean","last_fresh":"2023-06-27T12:44:47.510906+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:47.510906+0000","last_peered":"2023-06-27T12:44:47.510906+0000","last_clean":"2023-06-27T12:44:47.510906+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:47.510906+0000","last_undegraded":"2023-06-27T12:44:47.510906+0000","last_fullsized":"2023-06-27T12:44:47.510906+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"393'286","reported_seq":748,"reported_epoch":394,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:48.510197+0000","last_change":"2023-06-27T12:44:48.647681+0000","last_active":"2023-06-27T12:44:48.510197+0000","last_peered":"2023-06-27T12:44:48.510197+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:48.510197+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5597127,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":149,"num_read_kb":90270,"num_write":122,"num_write_kb":34526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":405,"reported_epoch":393,"state":"active+undersized","last_fresh":"2023-06-27T12:44:47.510251+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:47.510251+0000","last_peered":"2023-06-27T12:44:47.510251+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:47.510251+0000","last_undegraded":"2023-06-27T12:44:47.510251+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"382'238","reported_seq":1169,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:47.519311+0000","last_change":"2023-06-27T12:44:47.521686+0000","last_active":"2023-06-27T12:44:47.519311+0000","last_peered":"2023-06-27T12:44:47.519311+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:47.519311+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10673039,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":142738,"num_write":154,"num_write_kb":44067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"382'155","reported_seq":682,"reported_epoch":394,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:48.514938+0000","last_change":"2023-06-27T12:44:48.567702+0000","last_active":"2023-06-27T12:44:48.514938+0000","last_peered":"2023-06-27T12:44:48.514938+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:48.514938+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420373,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":62857,"num_write":62,"num_write_kb":17546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":662,"reported_epoch":393,"state":"active+undersized","last_fresh":"2023-06-27T12:44:47.503677+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:47.503677+0000","last_peered":"2023-06-27T12:44:47.503677+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:47.503677+0000","last_undegraded":"2023-06-27T12:44:47.503677+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":629,"reported_epoch":394,"state":"active+clean","last_fresh":"2023-06-27T12:44:48.513925+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:48.513925+0000","last_peered":"2023-06-27T12:44:48.513925+0000","last_clean":"2023-06-27T12:44:48.513925+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:48.513925+0000","last_undegraded":"2023-06-27T12:44:48.513925+0000","last_fullsized":"2023-06-27T12:44:48.513925+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"361'236","reported_seq":1147,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:47.515281+0000","last_change":"2023-06-27T12:44:47.520791+0000","last_active":"2023-06-27T12:44:47.515281+0000","last_peered":"2023-06-27T12:44:47.515281+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:47.515281+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5916328,"num_objects":26,"num_object_clones":23,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":124,"num_read_kb":135931,"num_write":125,"num_write_kb":37073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":102511899,"num_objects":412,"num_object_clones":365,"num_object_copies":1236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":2358,"num_read_kb":1743590,"num_write":2164,"num_write_kb":629571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3880,"ondisk_log_size":3880,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673375,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":323872,"kb_used_data":288856,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94047968,"statfs":{"total":96636764160,"available":96305119232,"internally_reserved":0,"allocated":295788544,"data_stored":345903038,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":335488,"kb_used_data":300472,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94036352,"statfs":{"total":96636764160,"available":96293224448,"internally_reserved":0,"allocated":307683328,"data_stored":353367190,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673375,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":362360,"kb_used_data":327344,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94009480,"statfs":{"total":96636764160,"available":96265707520,"internally_reserved":0,"allocated":335200256,"data_stored":387780874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:51.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.e 3.c 3.8 3.3 3.2 3.1 3.4 3.7 3.6 2023-06-27T12:44:51.417 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:51.399+0000 7f857492b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:51.420 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:51.416+0000 7f857492b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:51.526 INFO:tasks.rados.rados.0.smithi138.stdout:1624: done (3 left) 2023-06-27T12:44:51.526 INFO:tasks.rados.rados.0.smithi138.stdout:1625: read oid 23 snap 175 2023-06-27T12:44:51.526 INFO:tasks.rados.rados.0.smithi138.stdout:1625: expect deleted 2023-06-27T12:44:51.526 INFO:tasks.rados.rados.0.smithi138.stdout:1626: snap_remove snap 188 2023-06-27T12:44:51.628 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c,3.f] on osd.0 to force-backfill; instructing pg(s) [3.1,3.2,3.7,3.8] on osd.1 to force-backfill; instructing pg(s) [3.3,3.4,3.6,3.e] on osd.2 to force-backfill; 2023-06-27T12:44:51.629 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:44:51.892 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:52.022 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:52.016+0000 7fa92e121700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:52.024 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:52.021+0000 7fa92e121700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:52.070 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:52.073 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:52.073 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:52.528 INFO:tasks.rados.rados.0.smithi138.stdout:1626: done (4 left) 2023-06-27T12:44:52.528 INFO:tasks.rados.rados.0.smithi138.stdout:1627: snap_remove snap 187 2023-06-27T12:44:53.530 INFO:tasks.rados.rados.0.smithi138.stdout:1627: done (4 left) 2023-06-27T12:44:53.530 INFO:tasks.rados.rados.0.smithi138.stdout:1628: write (excl) oid 3 current snap is 191 2023-06-27T12:44:53.530 INFO:tasks.rados.rados.0.smithi138.stdout:1620: expect (ObjNum 763 snap 189 seq_num 763) 2023-06-27T12:44:53.721 INFO:tasks.rados.rados.0.smithi138.stdout:1622: finishing write tid 1 to smithi138138488-13 2023-06-27T12:44:53.722 INFO:tasks.rados.rados.0.smithi138.stdout:1621: finishing write tid 1 to smithi138138488-43 2023-06-27T12:44:53.722 INFO:tasks.rados.rados.0.smithi138.stdout:1622: finishing write tid 2 to smithi138138488-13 2023-06-27T12:44:53.722 INFO:tasks.rados.rados.0.smithi138.stdout:1622: finishing write tid 3 to smithi138138488-13 2023-06-27T12:44:53.722 INFO:tasks.rados.rados.0.smithi138.stdout:1622: finishing write tid 4 to smithi138138488-13 2023-06-27T12:44:53.722 INFO:tasks.rados.rados.0.smithi138.stdout:1622: oid 13 updating version 0 to 497 2023-06-27T12:44:53.722 INFO:tasks.rados.rados.0.smithi138.stdout:1622: oid 13 updating version 497 to 498 2023-06-27T12:44:53.723 INFO:tasks.rados.rados.0.smithi138.stdout:1622: oid 13 version 498 is already newer than 498 2023-06-27T12:44:53.723 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 498 (ObjNum 780 snap 191 seq_num 780) dirty exists 2023-06-27T12:44:53.723 INFO:tasks.rados.rados.0.smithi138.stdout:1622: left oid 13 (ObjNum 780 snap 191 seq_num 780) 2023-06-27T12:44:53.723 INFO:tasks.rados.rados.0.smithi138.stdout:1621: finishing write tid 2 to smithi138138488-43 2023-06-27T12:44:53.723 INFO:tasks.rados.rados.0.smithi138.stdout:1628: seq_num 782 ranges {743074=712724,2061154=794930} 2023-06-27T12:44:53.729 INFO:tasks.rados.rados.0.smithi138.stdout:1628: writing smithi138138488-3 from 743074 to 1455798 tid 1 2023-06-27T12:44:53.737 INFO:tasks.rados.rados.0.smithi138.stdout:1628: writing smithi138138488-3 from 2061154 to 2856084 tid 2 2023-06-27T12:44:53.737 INFO:tasks.rados.rados.0.smithi138.stdout:1620: done (4 left) 2023-06-27T12:44:53.737 INFO:tasks.rados.rados.0.smithi138.stdout:1622: done (3 left) 2023-06-27T12:44:53.737 INFO:tasks.rados.rados.0.smithi138.stdout:1629: snap_remove snap 190 2023-06-27T12:44:54.007 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~259s 2023-06-27T12:44:54.074 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:54.202 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:54.197+0000 7fb81e5a5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:54.205 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:54.201+0000 7fb81e5a5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:54.250 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:54.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:54.253 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:54.537 INFO:tasks.rados.rados.0.smithi138.stdout:1629: done (3 left) 2023-06-27T12:44:54.537 INFO:tasks.rados.rados.0.smithi138.stdout:1630: read oid 6 snap -1 2023-06-27T12:44:54.538 INFO:tasks.rados.rados.0.smithi138.stdout:1630: expect (ObjNum 768 snap 189 seq_num 768) 2023-06-27T12:44:54.538 INFO:tasks.rados.rados.0.smithi138.stdout:1631: snap_create 2023-06-27T12:44:54.538 INFO:tasks.rados.rados.0.smithi138.stdout:1621: finishing write tid 3 to smithi138138488-43 2023-06-27T12:44:54.538 INFO:tasks.rados.rados.0.smithi138.stdout:1621: finishing write tid 4 to smithi138138488-43 2023-06-27T12:44:54.538 INFO:tasks.rados.rados.0.smithi138.stdout:1621: finishing write tid 5 to smithi138138488-43 2023-06-27T12:44:54.538 INFO:tasks.rados.rados.0.smithi138.stdout:1621: oid 43 updating version 0 to 253 2023-06-27T12:44:54.538 INFO:tasks.rados.rados.0.smithi138.stdout:1621: oid 43 version 253 is already newer than 249 2023-06-27T12:44:54.539 INFO:tasks.rados.rados.0.smithi138.stdout:1621: oid 43 version 253 is already newer than 253 2023-06-27T12:44:54.539 INFO:tasks.rados.rados.0.smithi138.stdout:1621: oid 43 version 253 is already newer than 253 2023-06-27T12:44:54.539 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 253 (ObjNum 779 snap 191 seq_num 779) dirty exists 2023-06-27T12:44:54.539 INFO:tasks.rados.rados.0.smithi138.stdout:1621: left oid 43 (ObjNum 779 snap 191 seq_num 779) 2023-06-27T12:44:54.539 INFO:tasks.rados.rados.0.smithi138.stdout:1628: finishing write tid 1 to smithi138138488-3 2023-06-27T12:44:54.539 INFO:tasks.rados.rados.0.smithi138.stdout:1628: finishing write tid 2 to smithi138138488-3 2023-06-27T12:44:54.540 INFO:tasks.rados.rados.0.smithi138.stdout:1628: finishing write tid 3 to smithi138138488-3 2023-06-27T12:44:54.540 INFO:tasks.rados.rados.0.smithi138.stdout:1628: finishing write tid 4 to smithi138138488-3 2023-06-27T12:44:54.540 INFO:tasks.rados.rados.0.smithi138.stdout:1628: oid 3 updating version 0 to 260 2023-06-27T12:44:54.540 INFO:tasks.rados.rados.0.smithi138.stdout:1628: oid 3 updating version 260 to 263 2023-06-27T12:44:54.540 INFO:tasks.rados.rados.0.smithi138.stdout:1628: oid 3 version 263 is already newer than 263 2023-06-27T12:44:54.540 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 263 (ObjNum 782 snap 191 seq_num 782) dirty exists 2023-06-27T12:44:54.540 INFO:tasks.rados.rados.0.smithi138.stdout:1628: left oid 3 (ObjNum 782 snap 191 seq_num 782) 2023-06-27T12:44:54.579 INFO:tasks.rados.rados.0.smithi138.stdout:1630: expect (ObjNum 768 snap 189 seq_num 768) 2023-06-27T12:44:55.537 INFO:tasks.rados.rados.0.smithi138.stdout:1621: done (4 left) 2023-06-27T12:44:55.538 INFO:tasks.rados.rados.0.smithi138.stdout:1625: done (3 left) 2023-06-27T12:44:55.538 INFO:tasks.rados.rados.0.smithi138.stdout:1628: done (2 left) 2023-06-27T12:44:55.538 INFO:tasks.rados.rados.0.smithi138.stdout:1630: done (1 left) 2023-06-27T12:44:55.539 INFO:tasks.rados.rados.0.smithi138.stdout:1631: done (0 left) 2023-06-27T12:44:55.539 INFO:tasks.rados.rados.0.smithi138.stdout:1632: snap_remove snap 182 2023-06-27T12:44:56.254 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:44:56.394 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:56.389+0000 7f25dabb9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:56.397 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:56.393+0000 7f25dabb9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:56.445 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:56.446 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:56.544 INFO:tasks.rados.rados.0.smithi138.stdout:1632: done (0 left) 2023-06-27T12:44:56.545 INFO:tasks.rados.rados.0.smithi138.stdout:1633: write (excl) oid 35 current snap is 192 2023-06-27T12:44:56.545 INFO:tasks.rados.rados.0.smithi138.stdout:1633: seq_num 783 ranges {759881=787973,2119636=654941} 2023-06-27T12:44:56.555 INFO:tasks.rados.rados.0.smithi138.stdout:1633: writing smithi138138488-35 from 759881 to 1547854 tid 1 2023-06-27T12:44:56.564 INFO:tasks.rados.rados.0.smithi138.stdout:1633: writing smithi138138488-35 from 2119636 to 2774577 tid 2 2023-06-27T12:44:56.565 INFO:tasks.rados.rados.0.smithi138.stdout:1634: write (excl) oid 42 current snap is 192 2023-06-27T12:44:56.565 INFO:tasks.rados.rados.0.smithi138.stdout:1634: seq_num 784 ranges {776688=579575,1894471=798599} 2023-06-27T12:44:56.574 INFO:tasks.rados.rados.0.smithi138.stdout:1634: writing smithi138138488-42 from 776688 to 1356263 tid 1 2023-06-27T12:44:56.582 INFO:tasks.rados.rados.0.smithi138.stdout:1634: writing smithi138138488-42 from 1894471 to 2693070 tid 2 2023-06-27T12:44:56.582 INFO:tasks.rados.rados.0.smithi138.stdout:1635: write (excl) oid 9 current snap is 192 2023-06-27T12:44:56.582 INFO:tasks.rados.rados.0.smithi138.stdout:1635: seq_num 785 ranges {793495=654824,1952953=658610,3157167=36328} 2023-06-27T12:44:56.592 INFO:tasks.rados.rados.0.smithi138.stdout:1635: writing smithi138138488-9 from 793495 to 1448319 tid 1 2023-06-27T12:44:56.601 INFO:tasks.rados.rados.0.smithi138.stdout:1635: writing smithi138138488-9 from 1952953 to 2611563 tid 2 2023-06-27T12:44:56.604 INFO:tasks.rados.rados.0.smithi138.stdout:1635: writing smithi138138488-9 from 3157167 to 3193495 tid 3 2023-06-27T12:44:56.605 INFO:tasks.rados.rados.0.smithi138.stdout:1636: snap_remove snap 191 2023-06-27T12:44:56.642 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:44:56.643 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:44:56.643 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:44:56.783 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:56.776+0000 7f99b14d2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:56.786 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:56.782+0000 7f99b14d2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:56.977 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:44:56.979 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:44:56.991 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":606,"stamp":"2023-06-27T12:44:56.539411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":109033246,"num_objects":417,"num_object_clones":370,"num_object_copies":1251,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2396,"num_read_kb":1771413,"num_write":2191,"num_write_kb":637993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4034,"ondisk_log_size":4034,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1208820,"kb_used_data":1072852,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376278540,"statfs":{"total":386547056640,"available":385309224960,"internally_reserved":0,"allocated":1098600448,"data_stored":1273221284,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6521347,"num_objects":5,"num_object_clones":5,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":38,"num_read_kb":27823,"num_write":27,"num_write_kb":8422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016186"},"pg_stats":[{"pgid":"3.f","version":"387'58","reported_seq":948,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.556847+0000","last_change":"2023-06-27T12:44:53.561523+0000","last_active":"2023-06-27T12:44:53.556847+0000","last_peered":"2023-06-27T12:44:53.556847+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:53.556847+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2655671,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":6251,"num_write":37,"num_write_kb":10467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"396'287","reported_seq":1262,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.767601+0000","last_change":"2023-06-27T12:44:53.770212+0000","last_active":"2023-06-27T12:44:53.767601+0000","last_peered":"2023-06-27T12:44:53.767601+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:53.767601+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8067855,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":173,"num_read_kb":213041,"num_write":189,"num_write_kb":57277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":36}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"393'327","reported_seq":1243,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.973555+0000","last_change":"2023-06-27T12:44:53.974469+0000","last_active":"2023-06-27T12:44:53.973555+0000","last_peered":"2023-06-27T12:44:53.973555+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:53.973555+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11261251,"num_objects":37,"num_object_clones":32,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":276,"num_read_kb":186830,"num_write":205,"num_write_kb":59601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":37}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"393'230","reported_seq":1180,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.558813+0000","last_change":"2023-06-27T12:44:53.559364+0000","last_active":"2023-06-27T12:44:53.558813+0000","last_peered":"2023-06-27T12:44:53.558813+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:53.558813+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8336767,"num_objects":33,"num_object_clones":29,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":216,"num_read_kb":156199,"num_write":152,"num_write_kb":43433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":33}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":690,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.550160+0000","last_change":"2023-06-27T12:44:53.554835+0000","last_active":"2023-06-27T12:44:53.550160+0000","last_peered":"2023-06-27T12:44:53.550160+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:53.550160+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":91083,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"376'455","reported_seq":1249,"reported_epoch":399,"state":"active+clean","last_fresh":"2023-06-27T12:44:53.552135+0000","last_change":"2023-06-27T12:44:53.555431+0000","last_active":"2023-06-27T12:44:53.552135+0000","last_peered":"2023-06-27T12:44:53.552135+0000","last_clean":"2023-06-27T12:44:53.552135+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:53.552135+0000","last_undegraded":"2023-06-27T12:44:53.552135+0000","last_fullsized":"2023-06-27T12:44:53.552135+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10471388,"num_objects":49,"num_object_clones":43,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":333,"num_read_kb":211120,"num_write":265,"num_write_kb":75831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"393'265","reported_seq":1064,"reported_epoch":399,"state":"active+clean","last_fresh":"2023-06-27T12:44:53.557539+0000","last_change":"2023-06-27T12:44:53.557813+0000","last_active":"2023-06-27T12:44:53.557539+0000","last_peered":"2023-06-27T12:44:53.557539+0000","last_clean":"2023-06-27T12:44:53.557539+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:53.557539+0000","last_undegraded":"2023-06-27T12:44:53.557539+0000","last_fullsized":"2023-06-27T12:44:53.557539+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5428991,"num_objects":31,"num_object_clones":28,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":143,"num_read_kb":85556,"num_write":152,"num_write_kb":43826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"390'83","reported_seq":988,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.951467+0000","last_change":"2023-06-27T12:44:53.957720+0000","last_active":"2023-06-27T12:44:53.951467+0000","last_peered":"2023-06-27T12:44:53.951467+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:53.951467+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2672478,"num_objects":10,"num_object_clones":9,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":47129,"num_write":50,"num_write_kb":15046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":672,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-06-27T12:44:53.747033+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:53.747033+0000","last_peered":"2023-06-27T12:44:53.747033+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:53.747033+0000","last_undegraded":"2023-06-27T12:44:53.747033+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":892,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-06-27T12:44:53.953781+0000","last_change":"2023-06-27T12:44:53.967793+0000","last_active":"2023-06-27T12:44:53.953781+0000","last_peered":"2023-06-27T12:44:53.953781+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:53.953781+0000","last_undegraded":"2023-06-27T12:44:53.953781+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":647,"reported_epoch":399,"state":"active+clean","last_fresh":"2023-06-27T12:44:53.550701+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:53.550701+0000","last_peered":"2023-06-27T12:44:53.550701+0000","last_clean":"2023-06-27T12:44:53.550701+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:53.550701+0000","last_undegraded":"2023-06-27T12:44:53.550701+0000","last_fullsized":"2023-06-27T12:44:53.550701+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"393'220","reported_seq":1140,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.762640+0000","last_change":"2023-06-27T12:44:53.763520+0000","last_active":"2023-06-27T12:44:53.762640+0000","last_peered":"2023-06-27T12:44:53.762640+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:53.762640+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7681294,"num_objects":25,"num_object_clones":20,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":94,"num_read_kb":78897,"num_write":140,"num_write_kb":38532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":642,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-06-27T12:44:53.954949+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:53.954949+0000","last_peered":"2023-06-27T12:44:53.954949+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:53.954949+0000","last_undegraded":"2023-06-27T12:44:53.954949+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"396'498","reported_seq":833,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.773931+0000","last_change":"2023-06-27T12:44:53.775209+0000","last_active":"2023-06-27T12:44:53.773931+0000","last_peered":"2023-06-27T12:44:53.773931+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:53.773931+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8840977,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":85,"num_read_kb":60344,"num_write":153,"num_write_kb":47252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":639,"reported_epoch":399,"state":"active+clean","last_fresh":"2023-06-27T12:44:53.956926+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:53.956926+0000","last_peered":"2023-06-27T12:44:53.956926+0000","last_clean":"2023-06-27T12:44:53.956926+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:53.956926+0000","last_undegraded":"2023-06-27T12:44:53.956926+0000","last_fullsized":"2023-06-27T12:44:53.956926+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":399,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-06-27T12:44:53.547314+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:53.547314+0000","last_peered":"2023-06-27T12:44:53.547314+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:53.547314+0000","last_undegraded":"2023-06-27T12:44:53.547314+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"395'465","reported_seq":1377,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.977837+0000","last_change":"2023-06-27T12:44:53.978184+0000","last_active":"2023-06-27T12:44:53.977837+0000","last_peered":"2023-06-27T12:44:53.977837+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:53.977837+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10673105,"num_objects":56,"num_object_clones":50,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":338,"num_read_kb":203164,"num_write":313,"num_write_kb":91761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":56}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":647,"reported_epoch":399,"state":"active+clean","last_fresh":"2023-06-27T12:44:53.546353+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:53.546353+0000","last_peered":"2023-06-27T12:44:53.546353+0000","last_clean":"2023-06-27T12:44:53.546353+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:53.546353+0000","last_undegraded":"2023-06-27T12:44:53.546353+0000","last_fullsized":"2023-06-27T12:44:53.546353+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"393'286","reported_seq":763,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.973205+0000","last_change":"2023-06-27T12:44:53.973881+0000","last_active":"2023-06-27T12:44:53.973205+0000","last_peered":"2023-06-27T12:44:53.973205+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:53.973205+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5597127,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":149,"num_read_kb":90270,"num_write":122,"num_write_kb":34526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":415,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-06-27T12:44:53.744537+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:53.744537+0000","last_peered":"2023-06-27T12:44:53.744537+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:53.744537+0000","last_undegraded":"2023-06-27T12:44:53.744537+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"399'263","reported_seq":1192,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.791722+0000","last_change":"2023-06-27T12:44:53.791722+0000","last_active":"2023-06-27T12:44:53.791722+0000","last_peered":"2023-06-27T12:44:53.791722+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:53.791722+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12051246,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":142740,"num_write":163,"num_write_kb":46748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":32}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"382'155","reported_seq":695,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.949292+0000","last_change":"2023-06-27T12:44:53.958125+0000","last_active":"2023-06-27T12:44:53.949292+0000","last_peered":"2023-06-27T12:44:53.949292+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:53.949292+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420373,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":62857,"num_write":62,"num_write_kb":17546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":672,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-06-27T12:44:53.743521+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:53.743521+0000","last_peered":"2023-06-27T12:44:53.743521+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:53.743521+0000","last_undegraded":"2023-06-27T12:44:53.743521+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":639,"reported_epoch":399,"state":"active+clean","last_fresh":"2023-06-27T12:44:53.953380+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:53.953380+0000","last_peered":"2023-06-27T12:44:53.953380+0000","last_clean":"2023-06-27T12:44:53.953380+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:53.953380+0000","last_undegraded":"2023-06-27T12:44:53.953380+0000","last_fullsized":"2023-06-27T12:44:53.953380+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"395'241","reported_seq":1166,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:53.741504+0000","last_change":"2023-06-27T12:44:53.772734+0000","last_active":"2023-06-27T12:44:53.741504+0000","last_peered":"2023-06-27T12:44:53.741504+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:53.741504+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790325,"num_objects":27,"num_object_clones":24,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":126,"num_read_kb":135932,"num_write":130,"num_write_kb":38558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":27}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":109033246,"num_objects":417,"num_object_clones":370,"num_object_copies":1251,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2396,"num_read_kb":1771413,"num_write":2191,"num_write_kb":637993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3937,"ondisk_log_size":3937,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":327984,"kb_used_data":292968,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94043856,"statfs":{"total":96636764160,"available":96300908544,"internally_reserved":0,"allocated":299999232,"data_stored":351762043,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":335580,"kb_used_data":300564,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94036260,"statfs":{"total":96636764160,"available":96293130240,"internally_reserved":0,"allocated":307777536,"data_stored":354597529,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":363976,"kb_used_data":328960,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94007864,"statfs":{"total":96636764160,"available":96264052736,"internally_reserved":0,"allocated":336855040,"data_stored":389822160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:44:56.993 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.e 3.c 3.8 3.4 3.6 2023-06-27T12:44:57.125 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:57.112+0000 7efd13435700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:57.128 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:57.124+0000 7efd13435700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c,3.f] on osd.0 to force-backfill; instructing pg(s) [3.8] on osd.1 to force-backfill; instructing pg(s) [3.4,3.6,3.e] on osd.2 to force-backfill; 2023-06-27T12:44:57.321 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:44:57.550 INFO:tasks.rados.rados.0.smithi138.stdout:1636: done (3 left) 2023-06-27T12:44:57.551 INFO:tasks.rados.rados.0.smithi138.stdout:1637: write oid 49 current snap is 192 2023-06-27T12:44:57.551 INFO:tasks.rados.rados.0.smithi138.stdout:1633: finishing write tid 1 to smithi138138488-35 2023-06-27T12:44:57.551 INFO:tasks.rados.rados.0.smithi138.stdout:1634: finishing write tid 1 to smithi138138488-42 2023-06-27T12:44:57.551 INFO:tasks.rados.rados.0.smithi138.stdout:1633: finishing write tid 2 to smithi138138488-35 2023-06-27T12:44:57.551 INFO:tasks.rados.rados.0.smithi138.stdout:1637: seq_num 786 ranges {410302=730073,1895082=402268,2951884=258418} 2023-06-27T12:44:57.559 INFO:tasks.rados.rados.0.smithi138.stdout:1637: writing smithi138138488-49 from 410302 to 1140375 tid 1 2023-06-27T12:44:57.568 INFO:tasks.rados.rados.0.smithi138.stdout:1637: writing smithi138138488-49 from 1895082 to 2297350 tid 2 2023-06-27T12:44:57.574 INFO:tasks.rados.rados.0.smithi138.stdout:1637: writing smithi138138488-49 from 2951884 to 3210302 tid 3 2023-06-27T12:44:57.574 INFO:tasks.rados.rados.0.smithi138.stdout:1638: snap_remove snap 183 2023-06-27T12:44:58.447 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:44:58.558 INFO:tasks.rados.rados.0.smithi138.stdout:1638: done (4 left) 2023-06-27T12:44:58.558 INFO:tasks.rados.rados.0.smithi138.stdout:1639: read oid 7 snap -1 2023-06-27T12:44:58.558 INFO:tasks.rados.rados.0.smithi138.stdout:1639: expect (ObjNum 772 snap 189 seq_num 772) 2023-06-27T12:44:58.558 INFO:tasks.rados.rados.0.smithi138.stdout:1640: write oid 3 current snap is 192 2023-06-27T12:44:58.559 INFO:tasks.rados.rados.0.smithi138.stdout:1633: finishing write tid 3 to smithi138138488-35 2023-06-27T12:44:58.559 INFO:tasks.rados.rados.0.smithi138.stdout:1633: finishing write tid 4 to smithi138138488-35 2023-06-27T12:44:58.559 INFO:tasks.rados.rados.0.smithi138.stdout:1633: oid 35 updating version 0 to 506 2023-06-27T12:44:58.559 INFO:tasks.rados.rados.0.smithi138.stdout:1633: oid 35 version 506 is already newer than 506 2023-06-27T12:44:58.559 INFO:tasks.rados.rados.0.smithi138.stdout:1633: oid 35 version 506 is already newer than 503 2023-06-27T12:44:58.560 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 506 (ObjNum 783 snap 192 seq_num 783) dirty exists 2023-06-27T12:44:58.560 INFO:tasks.rados.rados.0.smithi138.stdout:1633: left oid 35 (ObjNum 783 snap 192 seq_num 783) 2023-06-27T12:44:58.560 INFO:tasks.rados.rados.0.smithi138.stdout:1634: finishing write tid 2 to smithi138138488-42 2023-06-27T12:44:58.560 INFO:tasks.rados.rados.0.smithi138.stdout:1634: finishing write tid 3 to smithi138138488-42 2023-06-27T12:44:58.560 INFO:tasks.rados.rados.0.smithi138.stdout:1640: seq_num 787 ranges {427109=405322,1553564=545926,2579307=647802} 2023-06-27T12:44:58.563 INFO:tasks.rados.rados.0.smithi138.stdout:1640: writing smithi138138488-3 from 427109 to 832431 tid 1 2023-06-27T12:44:58.573 INFO:tasks.rados.rados.0.smithi138.stdout:1640: writing smithi138138488-3 from 1553564 to 2099490 tid 2 2023-06-27T12:44:58.582 INFO:tasks.rados.rados.0.smithi138.stdout:1640: writing smithi138138488-3 from 2579307 to 3227109 tid 3 2023-06-27T12:44:58.583 INFO:tasks.rados.rados.0.smithi138.stdout:1633: done (5 left) 2023-06-27T12:44:58.583 INFO:tasks.rados.rados.0.smithi138.stdout:1641: write (excl) oid 35 current snap is 192 2023-06-27T12:44:58.583 INFO:tasks.rados.rados.0.smithi138.stdout:1641: seq_num 788 ranges {443916=480571,1612046=405937,2606730=495052} 2023-06-27T12:44:58.585 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:58.580+0000 7fbcc4c1f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:58.589 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:44:58.585+0000 7fbcc4c1f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:44:58.589 INFO:tasks.rados.rados.0.smithi138.stdout:1641: writing smithi138138488-35 from 443916 to 924487 tid 1 2023-06-27T12:44:58.597 INFO:tasks.rados.rados.0.smithi138.stdout:1641: writing smithi138138488-35 from 1612046 to 2017983 tid 2 2023-06-27T12:44:58.606 INFO:tasks.rados.rados.0.smithi138.stdout:1641: writing smithi138138488-35 from 2606730 to 3101782 tid 3 2023-06-27T12:44:58.606 INFO:tasks.rados.rados.0.smithi138.stdout:1642: write (excl) oid 13 current snap is 192 2023-06-27T12:44:58.606 INFO:tasks.rados.rados.0.smithi138.stdout:1634: finishing write tid 4 to smithi138138488-42 2023-06-27T12:44:58.606 INFO:tasks.rados.rados.0.smithi138.stdout:1634: oid 42 updating version 0 to 245 2023-06-27T12:44:58.606 INFO:tasks.rados.rados.0.smithi138.stdout:1634: oid 42 updating version 245 to 248 2023-06-27T12:44:58.606 INFO:tasks.rados.rados.0.smithi138.stdout:1634: oid 42 version 248 is already newer than 248 2023-06-27T12:44:58.607 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 42 v 248 (ObjNum 784 snap 192 seq_num 784) dirty exists 2023-06-27T12:44:58.607 INFO:tasks.rados.rados.0.smithi138.stdout:1634: left oid 42 (ObjNum 784 snap 192 seq_num 784) 2023-06-27T12:44:58.607 INFO:tasks.rados.rados.0.smithi138.stdout:1635: finishing write tid 1 to smithi138138488-9 2023-06-27T12:44:58.607 INFO:tasks.rados.rados.0.smithi138.stdout:1642: seq_num 789 ranges {460723=555820,1670528=549595,2634153=626570} 2023-06-27T12:44:58.613 INFO:tasks.rados.rados.0.smithi138.stdout:1642: writing smithi138138488-13 from 460723 to 1016543 tid 1 2023-06-27T12:44:58.623 INFO:tasks.rados.rados.0.smithi138.stdout:1642: writing smithi138138488-13 from 1670528 to 2220123 tid 2 2023-06-27T12:44:58.633 INFO:tasks.rados.rados.0.smithi138.stdout:1642: writing smithi138138488-13 from 2634153 to 3260723 tid 3 2023-06-27T12:44:58.633 INFO:tasks.rados.rados.0.smithi138.stdout:1634: done (6 left) 2023-06-27T12:44:58.633 INFO:tasks.rados.rados.0.smithi138.stdout:1643: read oid 44 snap -1 2023-06-27T12:44:58.633 INFO:tasks.rados.rados.0.smithi138.stdout:1643: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:44:58.634 INFO:tasks.rados.rados.0.smithi138.stdout:1644: snap_create 2023-06-27T12:44:58.634 INFO:tasks.rados.rados.0.smithi138.stdout:1635: finishing write tid 2 to smithi138138488-9 2023-06-27T12:44:58.634 INFO:tasks.rados.rados.0.smithi138.stdout:1635: finishing write tid 3 to smithi138138488-9 2023-06-27T12:44:58.634 INFO:tasks.rados.rados.0.smithi138.stdout:1635: finishing write tid 4 to smithi138138488-9 2023-06-27T12:44:58.634 INFO:tasks.rados.rados.0.smithi138.stdout:1635: finishing write tid 5 to smithi138138488-9 2023-06-27T12:44:58.634 INFO:tasks.rados.rados.0.smithi138.stdout:1635: oid 9 updating version 0 to 482 2023-06-27T12:44:58.634 INFO:tasks.rados.rados.0.smithi138.stdout:1635: oid 9 version 482 is already newer than 482 2023-06-27T12:44:58.634 INFO:tasks.rados.rados.0.smithi138.stdout:1635: oid 9 version 482 is already newer than 478 2023-06-27T12:44:58.635 INFO:tasks.rados.rados.0.smithi138.stdout:1635: oid 9 version 482 is already newer than 482 2023-06-27T12:44:58.635 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 482 (ObjNum 785 snap 192 seq_num 785) dirty exists 2023-06-27T12:44:58.635 INFO:tasks.rados.rados.0.smithi138.stdout:1635: left oid 9 (ObjNum 785 snap 192 seq_num 785) 2023-06-27T12:44:58.635 INFO:tasks.rados.rados.0.smithi138.stdout:1637: finishing write tid 1 to smithi138138488-49 2023-06-27T12:44:58.635 INFO:tasks.rados.rados.0.smithi138.stdout:1637: finishing write tid 2 to smithi138138488-49 2023-06-27T12:44:58.635 INFO:tasks.rados.rados.0.smithi138.stdout:1637: finishing write tid 3 to smithi138138488-49 2023-06-27T12:44:58.635 INFO:tasks.rados.rados.0.smithi138.stdout:1637: finishing write tid 4 to smithi138138488-49 2023-06-27T12:44:58.636 INFO:tasks.rados.rados.0.smithi138.stdout:1637: finishing write tid 5 to smithi138138488-49 2023-06-27T12:44:58.636 INFO:tasks.rados.rados.0.smithi138.stdout:1637: oid 49 updating version 0 to 253 2023-06-27T12:44:58.636 INFO:tasks.rados.rados.0.smithi138.stdout:1637: oid 49 version 253 is already newer than 252 2023-06-27T12:44:58.636 INFO:tasks.rados.rados.0.smithi138.stdout:1637: oid 49 version 253 is already newer than 253 2023-06-27T12:44:58.636 INFO:tasks.rados.rados.0.smithi138.stdout:1637: oid 49 version 253 is already newer than 253 2023-06-27T12:44:58.636 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 49 v 253 (ObjNum 786 snap 192 seq_num 786) dirty exists 2023-06-27T12:44:58.636 INFO:tasks.rados.rados.0.smithi138.stdout:1637: left oid 49 (ObjNum 786 snap 192 seq_num 786) 2023-06-27T12:44:58.636 INFO:tasks.rados.rados.0.smithi138.stdout:1641: finishing write tid 1 to smithi138138488-35 2023-06-27T12:44:58.639 INFO:tasks.rados.rados.0.smithi138.stdout:1641: finishing write tid 2 to smithi138138488-35 2023-06-27T12:44:58.672 INFO:tasks.rados.rados.0.smithi138.stdout:1643: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:44:58.727 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:44:58.730 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:44:58.731 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:44:58.879 INFO:tasks.rados.rados.0.smithi138.stdout:1639: expect (ObjNum 772 snap 189 seq_num 772) 2023-06-27T12:44:59.047 INFO:tasks.rados.rados.0.smithi138.stdout:1641: finishing write tid 3 to smithi138138488-35 2023-06-27T12:44:59.047 INFO:tasks.rados.rados.0.smithi138.stdout:1641: finishing write tid 4 to smithi138138488-35 2023-06-27T12:44:59.048 INFO:tasks.rados.rados.0.smithi138.stdout:1641: finishing write tid 5 to smithi138138488-35 2023-06-27T12:44:59.048 INFO:tasks.rados.rados.0.smithi138.stdout:1641: oid 35 updating version 0 to 512 2023-06-27T12:44:59.048 INFO:tasks.rados.rados.0.smithi138.stdout:1641: oid 35 version 512 is already newer than 509 2023-06-27T12:44:59.048 INFO:tasks.rados.rados.0.smithi138.stdout:1641: oid 35 version 512 is already newer than 512 2023-06-27T12:44:59.048 INFO:tasks.rados.rados.0.smithi138.stdout:1641: oid 35 version 512 is already newer than 512 2023-06-27T12:44:59.048 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 512 (ObjNum 788 snap 192 seq_num 788) dirty exists 2023-06-27T12:44:59.049 INFO:tasks.rados.rados.0.smithi138.stdout:1641: left oid 35 (ObjNum 788 snap 192 seq_num 788) 2023-06-27T12:44:59.049 INFO:tasks.rados.rados.0.smithi138.stdout:1640: finishing write tid 1 to smithi138138488-3 2023-06-27T12:44:59.049 INFO:tasks.rados.rados.0.smithi138.stdout:1640: finishing write tid 2 to smithi138138488-3 2023-06-27T12:44:59.049 INFO:tasks.rados.rados.0.smithi138.stdout:1640: finishing write tid 3 to smithi138138488-3 2023-06-27T12:44:59.049 INFO:tasks.rados.rados.0.smithi138.stdout:1640: finishing write tid 4 to smithi138138488-3 2023-06-27T12:44:59.049 INFO:tasks.rados.rados.0.smithi138.stdout:1642: finishing write tid 1 to smithi138138488-13 2023-06-27T12:44:59.049 INFO:tasks.rados.rados.0.smithi138.stdout:1640: finishing write tid 5 to smithi138138488-3 2023-06-27T12:44:59.050 INFO:tasks.rados.rados.0.smithi138.stdout:1640: oid 3 updating version 0 to 275 2023-06-27T12:44:59.050 INFO:tasks.rados.rados.0.smithi138.stdout:1640: oid 3 version 275 is already newer than 275 2023-06-27T12:44:59.050 INFO:tasks.rados.rados.0.smithi138.stdout:1640: oid 3 version 275 is already newer than 275 2023-06-27T12:44:59.050 INFO:tasks.rados.rados.0.smithi138.stdout:1640: oid 3 version 275 is already newer than 275 2023-06-27T12:44:59.050 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 275 (ObjNum 787 snap 192 seq_num 787) dirty exists 2023-06-27T12:44:59.050 INFO:tasks.rados.rados.0.smithi138.stdout:1640: left oid 3 (ObjNum 787 snap 192 seq_num 787) 2023-06-27T12:44:59.050 INFO:tasks.rados.rados.0.smithi138.stdout:1642: finishing write tid 2 to smithi138138488-13 2023-06-27T12:44:59.050 INFO:tasks.rados.rados.0.smithi138.stdout:1642: finishing write tid 3 to smithi138138488-13 2023-06-27T12:44:59.051 INFO:tasks.rados.rados.0.smithi138.stdout:1642: finishing write tid 4 to smithi138138488-13 2023-06-27T12:44:59.051 INFO:tasks.rados.rados.0.smithi138.stdout:1642: finishing write tid 5 to smithi138138488-13 2023-06-27T12:44:59.051 INFO:tasks.rados.rados.0.smithi138.stdout:1642: oid 13 updating version 0 to 516 2023-06-27T12:44:59.051 INFO:tasks.rados.rados.0.smithi138.stdout:1642: oid 13 version 516 is already newer than 516 2023-06-27T12:44:59.051 INFO:tasks.rados.rados.0.smithi138.stdout:1642: oid 13 version 516 is already newer than 516 2023-06-27T12:44:59.051 INFO:tasks.rados.rados.0.smithi138.stdout:1642: oid 13 version 516 is already newer than 513 2023-06-27T12:44:59.051 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 516 (ObjNum 789 snap 192 seq_num 789) dirty exists 2023-06-27T12:44:59.052 INFO:tasks.rados.rados.0.smithi138.stdout:1642: left oid 13 (ObjNum 789 snap 192 seq_num 789) 2023-06-27T12:44:59.512 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~264s 2023-06-27T12:44:59.565 INFO:tasks.rados.rados.0.smithi138.stdout:1635: done (7 left) 2023-06-27T12:44:59.566 INFO:tasks.rados.rados.0.smithi138.stdout:1637: done (6 left) 2023-06-27T12:44:59.566 INFO:tasks.rados.rados.0.smithi138.stdout:1639: done (5 left) 2023-06-27T12:44:59.566 INFO:tasks.rados.rados.0.smithi138.stdout:1640: done (4 left) 2023-06-27T12:44:59.566 INFO:tasks.rados.rados.0.smithi138.stdout:1641: done (3 left) 2023-06-27T12:44:59.566 INFO:tasks.rados.rados.0.smithi138.stdout:1642: done (2 left) 2023-06-27T12:44:59.566 INFO:tasks.rados.rados.0.smithi138.stdout:1643: done (1 left) 2023-06-27T12:44:59.567 INFO:tasks.rados.rados.0.smithi138.stdout:1644: done (0 left) 2023-06-27T12:44:59.567 INFO:tasks.rados.rados.0.smithi138.stdout:1645: read oid 26 snap -1 2023-06-27T12:44:59.567 INFO:tasks.rados.rados.0.smithi138.stdout:1645: expect (ObjNum 732 snap 178 seq_num 732) 2023-06-27T12:44:59.567 INFO:tasks.rados.rados.0.smithi138.stdout:1646: snap_create 2023-06-27T12:44:59.596 INFO:tasks.rados.rados.0.smithi138.stdout:1645: expect (ObjNum 732 snap 178 seq_num 732) 2023-06-27T12:45:00.473 INFO:tasks.rados.rados.0.smithi138.stdout:1645: done (1 left) 2023-06-27T12:45:00.473 INFO:tasks.rados.rados.0.smithi138.stdout:1646: done (0 left) 2023-06-27T12:45:00.473 INFO:tasks.rados.rados.0.smithi138.stdout:1647: read oid 19 snap -1 2023-06-27T12:45:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1647: expect (ObjNum 774 snap 190 seq_num 774) 2023-06-27T12:45:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1648: read oid 36 snap -1 2023-06-27T12:45:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1648: expect (ObjNum 700 snap 171 seq_num 700) 2023-06-27T12:45:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1649: write (excl) oid 25 current snap is 194 2023-06-27T12:45:00.474 INFO:tasks.rados.rados.0.smithi138.stdout:1649: seq_num 790 ranges {477530=631069,1612657=409606,2545223=517596} 2023-06-27T12:45:00.482 INFO:tasks.rados.rados.0.smithi138.stdout:1649: writing smithi138138488-25 from 477530 to 1108599 tid 1 2023-06-27T12:45:00.488 INFO:tasks.rados.rados.0.smithi138.stdout:1649: writing smithi138138488-25 from 1612657 to 2022263 tid 2 2023-06-27T12:45:00.497 INFO:tasks.rados.rados.0.smithi138.stdout:1649: writing smithi138138488-25 from 2545223 to 3062819 tid 3 2023-06-27T12:45:00.497 INFO:tasks.rados.rados.0.smithi138.stdout:1650: read oid 45 snap -1 2023-06-27T12:45:00.497 INFO:tasks.rados.rados.0.smithi138.stdout:1650: expect (ObjNum 764 snap 189 seq_num 764) 2023-06-27T12:45:00.498 INFO:tasks.rados.rados.0.smithi138.stdout:1651: snap_create 2023-06-27T12:45:00.522 INFO:tasks.rados.rados.0.smithi138.stdout:1647: expect (ObjNum 774 snap 190 seq_num 774) 2023-06-27T12:45:00.691 INFO:tasks.rados.rados.0.smithi138.stdout:1648: expect (ObjNum 700 snap 171 seq_num 700) 2023-06-27T12:45:00.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:00.864 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:00.859+0000 7f06b2b01700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:00.867 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:00.863+0000 7f06b2b01700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:00.913 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:00.917 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:00.917 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:00.939 INFO:tasks.rados.rados.0.smithi138.stdout:1650: expect (ObjNum 764 snap 189 seq_num 764) 2023-06-27T12:45:01.047 INFO:tasks.rados.rados.0.smithi138.stdout:1649: finishing write tid 1 to smithi138138488-25 2023-06-27T12:45:01.047 INFO:tasks.rados.rados.0.smithi138.stdout:1649: finishing write tid 2 to smithi138138488-25 2023-06-27T12:45:01.047 INFO:tasks.rados.rados.0.smithi138.stdout:1649: finishing write tid 3 to smithi138138488-25 2023-06-27T12:45:01.047 INFO:tasks.rados.rados.0.smithi138.stdout:1649: finishing write tid 4 to smithi138138488-25 2023-06-27T12:45:01.047 INFO:tasks.rados.rados.0.smithi138.stdout:1649: finishing write tid 5 to smithi138138488-25 2023-06-27T12:45:01.047 INFO:tasks.rados.rados.0.smithi138.stdout:1649: oid 25 updating version 0 to 170 2023-06-27T12:45:01.048 INFO:tasks.rados.rados.0.smithi138.stdout:1649: oid 25 updating version 170 to 174 2023-06-27T12:45:01.048 INFO:tasks.rados.rados.0.smithi138.stdout:1649: oid 25 version 174 is already newer than 174 2023-06-27T12:45:01.048 INFO:tasks.rados.rados.0.smithi138.stdout:1649: oid 25 version 174 is already newer than 174 2023-06-27T12:45:01.048 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 174 (ObjNum 790 snap 194 seq_num 790) dirty exists 2023-06-27T12:45:01.048 INFO:tasks.rados.rados.0.smithi138.stdout:1649: left oid 25 (ObjNum 790 snap 194 seq_num 790) 2023-06-27T12:45:01.478 INFO:tasks.rados.rados.0.smithi138.stdout:1647: done (4 left) 2023-06-27T12:45:01.478 INFO:tasks.rados.rados.0.smithi138.stdout:1648: done (3 left) 2023-06-27T12:45:01.479 INFO:tasks.rados.rados.0.smithi138.stdout:1649: done (2 left) 2023-06-27T12:45:01.479 INFO:tasks.rados.rados.0.smithi138.stdout:1650: done (1 left) 2023-06-27T12:45:01.479 INFO:tasks.rados.rados.0.smithi138.stdout:1651: done (0 left) 2023-06-27T12:45:01.479 INFO:tasks.rados.rados.0.smithi138.stdout:1652: read oid 33 snap -1 2023-06-27T12:45:01.479 INFO:tasks.rados.rados.0.smithi138.stdout:1652: expect (ObjNum 766 snap 189 seq_num 766) 2023-06-27T12:45:01.479 INFO:tasks.rados.rados.0.smithi138.stdout:1653: read oid 43 snap 192 2023-06-27T12:45:01.479 INFO:tasks.rados.rados.0.smithi138.stdout:1653: expect (ObjNum 779 snap 191 seq_num 779) 2023-06-27T12:45:01.480 INFO:tasks.rados.rados.0.smithi138.stdout:1654: delete oid 38 current snap is 195 2023-06-27T12:45:01.528 INFO:tasks.rados.rados.0.smithi138.stdout:1653: expect (ObjNum 779 snap 191 seq_num 779) 2023-06-27T12:45:01.735 INFO:tasks.rados.rados.0.smithi138.stdout:1652: expect (ObjNum 766 snap 189 seq_num 766) 2023-06-27T12:45:01.944 INFO:tasks.rados.rados.0.smithi138.stdout:1652: done (2 left) 2023-06-27T12:45:01.944 INFO:tasks.rados.rados.0.smithi138.stdout:1653: done (1 left) 2023-06-27T12:45:01.944 INFO:tasks.rados.rados.0.smithi138.stdout:1654: done (0 left) 2023-06-27T12:45:01.944 INFO:tasks.rados.rados.0.smithi138.stdout:1655: read oid 27 snap 192 2023-06-27T12:45:01.944 INFO:tasks.rados.rados.0.smithi138.stdout:1655: expect (ObjNum 778 snap 191 seq_num 778) 2023-06-27T12:45:01.944 INFO:tasks.rados.rados.0.smithi138.stdout:1656: write (excl) oid 36 current snap is 195 2023-06-27T12:45:01.945 INFO:tasks.rados.rados.0.smithi138.stdout:1656: seq_num 792 ranges {511144=497920,1445974=413275,2316422=656493} 2023-06-27T12:45:01.950 INFO:tasks.rados.rados.0.smithi138.stdout:1656: writing smithi138138488-36 from 511144 to 1009064 tid 1 2023-06-27T12:45:01.957 INFO:tasks.rados.rados.0.smithi138.stdout:1656: writing smithi138138488-36 from 1445974 to 1859249 tid 2 2023-06-27T12:45:01.965 INFO:tasks.rados.rados.0.smithi138.stdout:1656: writing smithi138138488-36 from 2316422 to 2972915 tid 3 2023-06-27T12:45:01.965 INFO:tasks.rados.rados.0.smithi138.stdout:1657: snap_remove snap 189 2023-06-27T12:45:02.333 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:45:02.333 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:45:02.333 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:45:02.462 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:02.455+0000 7f5f684c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:02.465 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:02.461+0000 7f5f684c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:02.576 INFO:tasks.rados.rados.0.smithi138.stdout:1657: done (2 left) 2023-06-27T12:45:02.576 INFO:tasks.rados.rados.0.smithi138.stdout:1658: delete oid 50 current snap is 195 2023-06-27T12:45:02.576 INFO:tasks.rados.rados.0.smithi138.stdout:1655: expect (ObjNum 778 snap 191 seq_num 778) 2023-06-27T12:45:02.685 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:45:02.687 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:45:02.699 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2023-06-27T12:45:02.571325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113285615,"num_objects":420,"num_object_clones":373,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":16,"num_read":2424,"num_read_kb":1786933,"num_write":2224,"num_write_kb":648034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4111,"ondisk_log_size":4111,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1224596,"kb_used_data":1085852,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":376262764,"statfs":{"total":386547056640,"available":385293070336,"internally_reserved":0,"allocated":1111912448,"data_stored":1286766790,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4252369,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":28,"num_read_kb":15520,"num_write":33,"num_write_kb":10041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020353"},"pg_stats":[{"pgid":"3.f","version":"387'58","reported_seq":962,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.577383+0000","last_change":"2023-06-27T12:44:58.577723+0000","last_active":"2023-06-27T12:44:58.577383+0000","last_peered":"2023-06-27T12:44:58.577383+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:44:58.577383+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2655671,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":6251,"num_write":37,"num_write_kb":10467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"396'287","reported_seq":1276,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.595557+0000","last_change":"2023-06-27T12:44:58.596013+0000","last_active":"2023-06-27T12:44:58.595557+0000","last_peered":"2023-06-27T12:44:58.595557+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:44:58.595557+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8067855,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":173,"num_read_kb":213041,"num_write":189,"num_write_kb":57277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":36}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"393'327","reported_seq":1257,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.855891+0000","last_change":"2023-06-27T12:44:58.856738+0000","last_active":"2023-06-27T12:44:58.855891+0000","last_peered":"2023-06-27T12:44:58.855891+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:44:58.855891+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11261251,"num_objects":37,"num_object_clones":32,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":276,"num_read_kb":186830,"num_write":205,"num_write_kb":59601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":37}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"402'248","reported_seq":1198,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.604742+0000","last_change":"2023-06-27T12:44:58.611443+0000","last_active":"2023-06-27T12:44:58.604742+0000","last_peered":"2023-06-27T12:44:58.604742+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:44:58.604742+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874591,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":218,"num_read_kb":156200,"num_write":156,"num_write_kb":44779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"345'104","reported_seq":702,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.587459+0000","last_change":"2023-06-27T12:44:58.594959+0000","last_active":"2023-06-27T12:44:58.587459+0000","last_peered":"2023-06-27T12:44:58.587459+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:44:58.587459+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084398,"num_objects":9,"num_object_clones":8,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":91083,"num_write":58,"num_write_kb":17589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"376'455","reported_seq":1264,"reported_epoch":404,"state":"active+clean","last_fresh":"2023-06-27T12:44:58.660909+0000","last_change":"2023-06-27T12:44:58.595543+0000","last_active":"2023-06-27T12:44:58.660909+0000","last_peered":"2023-06-27T12:44:58.660909+0000","last_clean":"2023-06-27T12:44:58.660909+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:44:58.660909+0000","last_undegraded":"2023-06-27T12:44:58.660909+0000","last_fullsized":"2023-06-27T12:44:58.660909+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10471388,"num_objects":49,"num_object_clones":43,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":340,"num_read_kb":218235,"num_write":265,"num_write_kb":75831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"393'265","reported_seq":1081,"reported_epoch":404,"state":"active+clean","last_fresh":"2023-06-27T12:44:58.696711+0000","last_change":"2023-06-27T12:44:58.697578+0000","last_active":"2023-06-27T12:44:58.696711+0000","last_peered":"2023-06-27T12:44:58.696711+0000","last_clean":"2023-06-27T12:44:58.696711+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:44:58.696711+0000","last_undegraded":"2023-06-27T12:44:58.696711+0000","last_fullsized":"2023-06-27T12:44:58.696711+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5428991,"num_objects":31,"num_object_clones":28,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":150,"num_read_kb":91286,"num_write":152,"num_write_kb":43826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"390'83","reported_seq":1002,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.836476+0000","last_change":"2023-06-27T12:44:58.836816+0000","last_active":"2023-06-27T12:44:58.836476+0000","last_peered":"2023-06-27T12:44:58.836476+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:44:58.836476+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2672478,"num_objects":10,"num_object_clones":9,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":47129,"num_write":50,"num_write_kb":15046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":680,"reported_epoch":404,"state":"active+undersized","last_fresh":"2023-06-27T12:44:58.566115+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:44:58.566115+0000","last_peered":"2023-06-27T12:44:58.566115+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:44:58.566115+0000","last_undegraded":"2023-06-27T12:44:58.566115+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":904,"reported_epoch":404,"state":"active+undersized","last_fresh":"2023-06-27T12:44:58.832767+0000","last_change":"2023-06-27T12:44:58.852587+0000","last_active":"2023-06-27T12:44:58.832767+0000","last_peered":"2023-06-27T12:44:58.832767+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:44:58.832767+0000","last_undegraded":"2023-06-27T12:44:58.832767+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":655,"reported_epoch":404,"state":"active+clean","last_fresh":"2023-06-27T12:44:58.572945+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:44:58.572945+0000","last_peered":"2023-06-27T12:44:58.572945+0000","last_clean":"2023-06-27T12:44:58.572945+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:44:58.572945+0000","last_undegraded":"2023-06-27T12:44:58.572945+0000","last_fullsized":"2023-06-27T12:44:58.572945+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"393'220","reported_seq":1156,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.591173+0000","last_change":"2023-06-27T12:44:58.592156+0000","last_active":"2023-06-27T12:44:58.591173+0000","last_peered":"2023-06-27T12:44:58.591173+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:44:58.591173+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7681294,"num_objects":25,"num_object_clones":20,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":96,"num_read_kb":81565,"num_write":140,"num_write_kb":38532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":25}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":650,"reported_epoch":404,"state":"active+undersized","last_fresh":"2023-06-27T12:44:58.837718+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:44:58.837718+0000","last_peered":"2023-06-27T12:44:58.837718+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:44:58.837718+0000","last_undegraded":"2023-06-27T12:44:58.837718+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"404'516","reported_seq":861,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.731315+0000","last_change":"2023-06-27T12:44:58.588143+0000","last_active":"2023-06-27T12:44:58.731315+0000","last_peered":"2023-06-27T12:44:58.731315+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:44:58.731315+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874624,"num_objects":31,"num_object_clones":29,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":91,"num_read_kb":60347,"num_write":167,"num_write_kb":51705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":31}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":647,"reported_epoch":404,"state":"active+clean","last_fresh":"2023-06-27T12:44:58.830313+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:44:58.830313+0000","last_peered":"2023-06-27T12:44:58.830313+0000","last_clean":"2023-06-27T12:44:58.830313+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:44:58.830313+0000","last_undegraded":"2023-06-27T12:44:58.830313+0000","last_fullsized":"2023-06-27T12:44:58.830313+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":407,"reported_epoch":404,"state":"active+undersized","last_fresh":"2023-06-27T12:44:58.575786+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:44:58.575786+0000","last_peered":"2023-06-27T12:44:58.575786+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:44:58.575786+0000","last_undegraded":"2023-06-27T12:44:58.575786+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"402'482","reported_seq":1396,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.857606+0000","last_change":"2023-06-27T12:44:58.857933+0000","last_active":"2023-06-27T12:44:58.857606+0000","last_peered":"2023-06-27T12:44:58.857606+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:44:58.857606+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11059666,"num_objects":57,"num_object_clones":51,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":340,"num_read_kb":203165,"num_write":318,"num_write_kb":93081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":57}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":655,"reported_epoch":404,"state":"active+clean","last_fresh":"2023-06-27T12:44:58.593209+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:44:58.593209+0000","last_peered":"2023-06-27T12:44:58.593209+0000","last_clean":"2023-06-27T12:44:58.593209+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:44:58.593209+0000","last_undegraded":"2023-06-27T12:44:58.593209+0000","last_fullsized":"2023-06-27T12:44:58.593209+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"393'286","reported_seq":777,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.855051+0000","last_change":"2023-06-27T12:44:58.855427+0000","last_active":"2023-06-27T12:44:58.855051+0000","last_peered":"2023-06-27T12:44:58.855051+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:44:58.855051+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5597127,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":149,"num_read_kb":90270,"num_write":122,"num_write_kb":34526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":423,"reported_epoch":404,"state":"active+undersized","last_fresh":"2023-06-27T12:44:58.568441+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:44:58.568441+0000","last_peered":"2023-06-27T12:44:58.568441+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:44:58.568441+0000","last_undegraded":"2023-06-27T12:44:58.568441+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"404'275","reported_seq":1211,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.656409+0000","last_change":"2023-06-27T12:44:58.649409+0000","last_active":"2023-06-27T12:44:58.656409+0000","last_peered":"2023-06-27T12:44:58.656409+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:44:58.656409+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12135281,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":142741,"num_write":168,"num_write_kb":48311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":32}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"382'155","reported_seq":707,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.827865+0000","last_change":"2023-06-27T12:44:58.856808+0000","last_active":"2023-06-27T12:44:58.827865+0000","last_peered":"2023-06-27T12:44:58.827865+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:44:58.827865+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420373,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":110,"num_read_kb":62857,"num_write":62,"num_write_kb":17546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":680,"reported_epoch":404,"state":"active+undersized","last_fresh":"2023-06-27T12:44:58.565328+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:44:58.565328+0000","last_peered":"2023-06-27T12:44:58.565328+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:44:58.565328+0000","last_undegraded":"2023-06-27T12:44:58.565328+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":647,"reported_epoch":404,"state":"active+clean","last_fresh":"2023-06-27T12:44:58.832260+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:44:58.832260+0000","last_peered":"2023-06-27T12:44:58.832260+0000","last_clean":"2023-06-27T12:44:58.832260+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:44:58.832260+0000","last_undegraded":"2023-06-27T12:44:58.832260+0000","last_fullsized":"2023-06-27T12:44:58.832260+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"403'253","reported_seq":1183,"reported_epoch":404,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:44:58.562784+0000","last_change":"2023-06-27T12:44:58.603234+0000","last_active":"2023-06-27T12:44:58.562784+0000","last_peered":"2023-06-27T12:44:58.562784+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:44:58.562784+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10000627,"num_objects":27,"num_object_clones":24,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":135933,"num_write":135,"num_write_kb":39917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":27}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":113285615,"num_objects":420,"num_object_clones":373,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":16,"num_read":2424,"num_read_kb":1786933,"num_write":2224,"num_write_kb":648034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4014,"ondisk_log_size":4014,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673385,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":334932,"kb_used_data":298528,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94036908,"statfs":{"total":96636764160,"available":96293793792,"internally_reserved":0,"allocated":305692672,"data_stored":356727820,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706090,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":338028,"kb_used_data":302616,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94033812,"statfs":{"total":96636764160,"available":96290623488,"internally_reserved":0,"allocated":309878784,"data_stored":356685949,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673385,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":370356,"kb_used_data":334348,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94001484,"statfs":{"total":96636764160,"available":96257519616,"internally_reserved":0,"allocated":342372352,"data_stored":396313469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:45:02.701 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.e 3.c 3.3 3.0 3.2 3.1 3.4 3.6 2023-06-27T12:45:02.721 INFO:tasks.rados.rados.0.smithi138.stdout:1656: finishing write tid 1 to smithi138138488-36 2023-06-27T12:45:02.721 INFO:tasks.rados.rados.0.smithi138.stdout:1655: done (2 left) 2023-06-27T12:45:02.721 INFO:tasks.rados.rados.0.smithi138.stdout:1658: done (1 left) 2023-06-27T12:45:02.721 INFO:tasks.rados.rados.0.smithi138.stdout:1659: write oid 41 current snap is 195 2023-06-27T12:45:02.721 INFO:tasks.rados.rados.0.smithi138.stdout:1656: finishing write tid 2 to smithi138138488-36 2023-06-27T12:45:02.722 INFO:tasks.rados.rados.0.smithi138.stdout:1656: finishing write tid 3 to smithi138138488-36 2023-06-27T12:45:02.722 INFO:tasks.rados.rados.0.smithi138.stdout:1656: finishing write tid 4 to smithi138138488-36 2023-06-27T12:45:02.722 INFO:tasks.rados.rados.0.smithi138.stdout:1659: seq_num 794 ranges {544758=648418,1846585=416944,2771268=573490} 2023-06-27T12:45:02.727 INFO:tasks.rados.rados.0.smithi138.stdout:1659: writing smithi138138488-41 from 544758 to 1193176 tid 1 2023-06-27T12:45:02.733 INFO:tasks.rados.rados.0.smithi138.stdout:1659: writing smithi138138488-41 from 1846585 to 2263529 tid 2 2023-06-27T12:45:02.740 INFO:tasks.rados.rados.0.smithi138.stdout:1659: writing smithi138138488-41 from 2771268 to 3344758 tid 3 2023-06-27T12:45:02.740 INFO:tasks.rados.rados.0.smithi138.stdout:1660: read oid 23 snap -1 2023-06-27T12:45:02.740 INFO:tasks.rados.rados.0.smithi138.stdout:1660: expect (ObjNum 716 snap 178 seq_num 716) 2023-06-27T12:45:02.740 INFO:tasks.rados.rados.0.smithi138.stdout:1661: snap_create 2023-06-27T12:45:02.740 INFO:tasks.rados.rados.0.smithi138.stdout:1656: finishing write tid 5 to smithi138138488-36 2023-06-27T12:45:02.740 INFO:tasks.rados.rados.0.smithi138.stdout:1656: oid 36 updating version 0 to 337 2023-06-27T12:45:02.741 INFO:tasks.rados.rados.0.smithi138.stdout:1656: oid 36 updating version 337 to 341 2023-06-27T12:45:02.741 INFO:tasks.rados.rados.0.smithi138.stdout:1656: oid 36 version 341 is already newer than 341 2023-06-27T12:45:02.741 INFO:tasks.rados.rados.0.smithi138.stdout:1656: oid 36 version 341 is already newer than 341 2023-06-27T12:45:02.741 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 341 (ObjNum 792 snap 195 seq_num 792) dirty exists 2023-06-27T12:45:02.741 INFO:tasks.rados.rados.0.smithi138.stdout:1656: left oid 36 (ObjNum 792 snap 195 seq_num 792) 2023-06-27T12:45:02.784 INFO:tasks.rados.rados.0.smithi138.stdout:1659: finishing write tid 1 to smithi138138488-41 2023-06-27T12:45:02.789 INFO:tasks.rados.rados.0.smithi138.stdout:1659: finishing write tid 2 to smithi138138488-41 2023-06-27T12:45:02.798 INFO:tasks.rados.rados.0.smithi138.stdout:1660: expect (ObjNum 716 snap 178 seq_num 716) 2023-06-27T12:45:02.841 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:02.825+0000 7f8f18752700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:02.843 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:02.840+0000 7f8f18752700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:02.879 INFO:tasks.rados.rados.0.smithi138.stdout:1659: finishing write tid 3 to smithi138138488-41 2023-06-27T12:45:02.879 INFO:tasks.rados.rados.0.smithi138.stdout:1659: finishing write tid 4 to smithi138138488-41 2023-06-27T12:45:02.879 INFO:tasks.rados.rados.0.smithi138.stdout:1659: finishing write tid 5 to smithi138138488-41 2023-06-27T12:45:02.879 INFO:tasks.rados.rados.0.smithi138.stdout:1659: oid 41 updating version 0 to 280 2023-06-27T12:45:02.879 INFO:tasks.rados.rados.0.smithi138.stdout:1659: oid 41 version 280 is already newer than 280 2023-06-27T12:45:02.880 INFO:tasks.rados.rados.0.smithi138.stdout:1659: oid 41 version 280 is already newer than 280 2023-06-27T12:45:02.880 INFO:tasks.rados.rados.0.smithi138.stdout:1659: oid 41 version 280 is already newer than 280 2023-06-27T12:45:02.880 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 41 v 280 (ObjNum 794 snap 195 seq_num 794) dirty exists 2023-06-27T12:45:02.880 INFO:tasks.rados.rados.0.smithi138.stdout:1659: left oid 41 (ObjNum 794 snap 195 seq_num 794) 2023-06-27T12:45:02.918 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:45:03.044 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c] on osd.0 to force-backfill; instructing pg(s) [3.1,3.2] on osd.1 to force-backfill; instructing pg(s) [3.0,3.3,3.4,3.6,3.e] on osd.2 to force-backfill; 2023-06-27T12:45:03.045 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:45:03.045 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:03.040+0000 7f0048e4c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:03.049 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:03.045+0000 7f0048e4c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:03.098 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:03.098 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:03.578 INFO:tasks.rados.rados.0.smithi138.stdout:1656: done (3 left) 2023-06-27T12:45:03.579 INFO:tasks.rados.rados.0.smithi138.stdout:1659: done (2 left) 2023-06-27T12:45:03.579 INFO:tasks.rados.rados.0.smithi138.stdout:1660: done (1 left) 2023-06-27T12:45:03.580 INFO:tasks.rados.rados.0.smithi138.stdout:1661: done (0 left) 2023-06-27T12:45:03.580 INFO:tasks.rados.rados.0.smithi138.stdout:1662: read oid 1 snap -1 2023-06-27T12:45:03.580 INFO:tasks.rados.rados.0.smithi138.stdout:1662: expect (ObjNum 742 snap 182 seq_num 742) 2023-06-27T12:45:03.581 INFO:tasks.rados.rados.0.smithi138.stdout:1663: write (excl) oid 31 current snap is 196 2023-06-27T12:45:03.581 INFO:tasks.rados.rados.0.smithi138.stdout:1663: seq_num 795 ranges {561565=723667,1905067=560602,3082338=279227} 2023-06-27T12:45:03.588 INFO:tasks.rados.rados.0.smithi138.stdout:1663: writing smithi138138488-31 from 561565 to 1285232 tid 1 2023-06-27T12:45:03.597 INFO:tasks.rados.rados.0.smithi138.stdout:1663: writing smithi138138488-31 from 1905067 to 2465669 tid 2 2023-06-27T12:45:03.603 INFO:tasks.rados.rados.0.smithi138.stdout:1663: writing smithi138138488-31 from 3082338 to 3361565 tid 3 2023-06-27T12:45:03.603 INFO:tasks.rados.rados.0.smithi138.stdout:1664: write oid 10 current snap is 196 2023-06-27T12:45:03.604 INFO:tasks.rados.rados.0.smithi138.stdout:1664: seq_num 796 ranges {578372=798916,1963549=420613,2826114=417934} 2023-06-27T12:45:03.614 INFO:tasks.rados.rados.0.smithi138.stdout:1664: writing smithi138138488-10 from 578372 to 1377288 tid 1 2023-06-27T12:45:03.621 INFO:tasks.rados.rados.0.smithi138.stdout:1664: writing smithi138138488-10 from 1963549 to 2384162 tid 2 2023-06-27T12:45:03.627 INFO:tasks.rados.rados.0.smithi138.stdout:1664: writing smithi138138488-10 from 2826114 to 3244048 tid 3 2023-06-27T12:45:03.628 INFO:tasks.rados.rados.0.smithi138.stdout:1665: write oid 50 current snap is 196 2023-06-27T12:45:03.628 INFO:tasks.rados.rados.0.smithi138.stdout:1665: seq_num 797 ranges {595179=474165,1622031=564271,2737184=629206} 2023-06-27T12:45:03.635 INFO:tasks.rados.rados.0.smithi138.stdout:1665: writing smithi138138488-50 from 595179 to 1069344 tid 1 2023-06-27T12:45:03.644 INFO:tasks.rados.rados.0.smithi138.stdout:1665: writing smithi138138488-50 from 1622031 to 2186302 tid 2 2023-06-27T12:45:03.654 INFO:tasks.rados.rados.0.smithi138.stdout:1665: writing smithi138138488-50 from 2737184 to 3366390 tid 3 2023-06-27T12:45:03.654 INFO:tasks.rados.rados.0.smithi138.stdout:1666: write oid 38 current snap is 196 2023-06-27T12:45:03.654 INFO:tasks.rados.rados.0.smithi138.stdout:1666: seq_num 798 ranges {611986=549414,1564160=424282,2764607=440478} 2023-06-27T12:45:03.663 INFO:tasks.rados.rados.0.smithi138.stdout:1666: writing smithi138138488-38 from 611986 to 1161400 tid 1 2023-06-27T12:45:03.670 INFO:tasks.rados.rados.0.smithi138.stdout:1666: writing smithi138138488-38 from 1564160 to 1988442 tid 2 2023-06-27T12:45:03.679 INFO:tasks.rados.rados.0.smithi138.stdout:1666: writing smithi138138488-38 from 2764607 to 3205085 tid 3 2023-06-27T12:45:03.679 INFO:tasks.rados.rados.0.smithi138.stdout:1667: read oid 37 snap 192 2023-06-27T12:45:03.680 INFO:tasks.rados.rados.0.smithi138.stdout:1667: expect (ObjNum 771 snap 189 seq_num 771) 2023-06-27T12:45:03.680 INFO:tasks.rados.rados.0.smithi138.stdout:1668: write oid 28 current snap is 196 2023-06-27T12:45:03.680 INFO:tasks.rados.rados.0.smithi138.stdout:1668: seq_num 799 ranges {628793=741016,2138995=567940,3192030=236763} 2023-06-27T12:45:03.690 INFO:tasks.rados.rados.0.smithi138.stdout:1668: writing smithi138138488-28 from 628793 to 1369809 tid 1 2023-06-27T12:45:03.700 INFO:tasks.rados.rados.0.smithi138.stdout:1668: writing smithi138138488-28 from 2138995 to 2706935 tid 2 2023-06-27T12:45:03.706 INFO:tasks.rados.rados.0.smithi138.stdout:1668: writing smithi138138488-28 from 3192030 to 3428793 tid 3 2023-06-27T12:45:03.706 INFO:tasks.rados.rados.0.smithi138.stdout:1669: read oid 5 snap -1 2023-06-27T12:45:03.707 INFO:tasks.rados.rados.0.smithi138.stdout:1669: expect (ObjNum 731 snap 178 seq_num 731) 2023-06-27T12:45:03.707 INFO:tasks.rados.rados.0.smithi138.stdout:1670: read oid 13 snap -1 2023-06-27T12:45:03.707 INFO:tasks.rados.rados.0.smithi138.stdout:1670: expect (ObjNum 789 snap 192 seq_num 789) 2023-06-27T12:45:03.707 INFO:tasks.rados.rados.0.smithi138.stdout:1671: snap_create 2023-06-27T12:45:03.707 INFO:tasks.rados.rados.0.smithi138.stdout:1662: expect (ObjNum 742 snap 182 seq_num 742) 2023-06-27T12:45:03.813 INFO:tasks.rados.rados.0.smithi138.stdout:1665: finishing write tid 1 to smithi138138488-50 2023-06-27T12:45:03.813 INFO:tasks.rados.rados.0.smithi138.stdout:1663: finishing write tid 1 to smithi138138488-31 2023-06-27T12:45:03.814 INFO:tasks.rados.rados.0.smithi138.stdout:1665: finishing write tid 2 to smithi138138488-50 2023-06-27T12:45:03.814 INFO:tasks.rados.rados.0.smithi138.stdout:1664: finishing write tid 1 to smithi138138488-10 2023-06-27T12:45:03.814 INFO:tasks.rados.rados.0.smithi138.stdout:1664: finishing write tid 2 to smithi138138488-10 2023-06-27T12:45:03.814 INFO:tasks.rados.rados.0.smithi138.stdout:1664: finishing write tid 3 to smithi138138488-10 2023-06-27T12:45:03.815 INFO:tasks.rados.rados.0.smithi138.stdout:1665: finishing write tid 3 to smithi138138488-50 2023-06-27T12:45:03.833 INFO:tasks.rados.rados.0.smithi138.stdout:1663: finishing write tid 2 to smithi138138488-31 2023-06-27T12:45:03.836 INFO:tasks.rados.rados.0.smithi138.stdout:1665: finishing write tid 4 to smithi138138488-50 2023-06-27T12:45:03.837 INFO:tasks.rados.rados.0.smithi138.stdout:1667: expect (ObjNum 771 snap 189 seq_num 771) 2023-06-27T12:45:03.987 INFO:tasks.rados.rados.0.smithi138.stdout:1665: finishing write tid 5 to smithi138138488-50 2023-06-27T12:45:03.987 INFO:tasks.rados.rados.0.smithi138.stdout:1665: oid 50 updating version 0 to 95 2023-06-27T12:45:03.988 INFO:tasks.rados.rados.0.smithi138.stdout:1665: oid 50 version 95 is already newer than 93 2023-06-27T12:45:03.988 INFO:tasks.rados.rados.0.smithi138.stdout:1665: oid 50 version 95 is already newer than 95 2023-06-27T12:45:03.988 INFO:tasks.rados.rados.0.smithi138.stdout:1665: oid 50 version 95 is already newer than 93 2023-06-27T12:45:03.988 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 50 v 95 (ObjNum 797 snap 196 seq_num 797) dirty exists 2023-06-27T12:45:03.988 INFO:tasks.rados.rados.0.smithi138.stdout:1665: left oid 50 (ObjNum 797 snap 196 seq_num 797) 2023-06-27T12:45:03.988 INFO:tasks.rados.rados.0.smithi138.stdout:1669: expect (ObjNum 731 snap 178 seq_num 731) 2023-06-27T12:45:04.114 INFO:tasks.rados.rados.0.smithi138.stdout:1670: expect (ObjNum 789 snap 192 seq_num 789) 2023-06-27T12:45:04.270 INFO:tasks.rados.rados.0.smithi138.stdout:1664: finishing write tid 4 to smithi138138488-10 2023-06-27T12:45:04.271 INFO:tasks.rados.rados.0.smithi138.stdout:1663: finishing write tid 3 to smithi138138488-31 2023-06-27T12:45:04.271 INFO:tasks.rados.rados.0.smithi138.stdout:1663: finishing write tid 4 to smithi138138488-31 2023-06-27T12:45:04.271 INFO:tasks.rados.rados.0.smithi138.stdout:1664: finishing write tid 5 to smithi138138488-10 2023-06-27T12:45:04.271 INFO:tasks.rados.rados.0.smithi138.stdout:1664: oid 10 updating version 0 to 285 2023-06-27T12:45:04.271 INFO:tasks.rados.rados.0.smithi138.stdout:1664: oid 10 version 285 is already newer than 285 2023-06-27T12:45:04.272 INFO:tasks.rados.rados.0.smithi138.stdout:1664: oid 10 version 285 is already newer than 285 2023-06-27T12:45:04.272 INFO:tasks.rados.rados.0.smithi138.stdout:1664: oid 10 version 285 is already newer than 285 2023-06-27T12:45:04.272 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 285 (ObjNum 796 snap 196 seq_num 796) dirty exists 2023-06-27T12:45:04.272 INFO:tasks.rados.rados.0.smithi138.stdout:1664: left oid 10 (ObjNum 796 snap 196 seq_num 796) 2023-06-27T12:45:04.272 INFO:tasks.rados.rados.0.smithi138.stdout:1663: finishing write tid 5 to smithi138138488-31 2023-06-27T12:45:04.272 INFO:tasks.rados.rados.0.smithi138.stdout:1663: oid 31 updating version 0 to 464 2023-06-27T12:45:04.272 INFO:tasks.rados.rados.0.smithi138.stdout:1663: oid 31 version 464 is already newer than 464 2023-06-27T12:45:04.272 INFO:tasks.rados.rados.0.smithi138.stdout:1663: oid 31 version 464 is already newer than 460 2023-06-27T12:45:04.273 INFO:tasks.rados.rados.0.smithi138.stdout:1663: oid 31 version 464 is already newer than 464 2023-06-27T12:45:04.273 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 464 (ObjNum 795 snap 196 seq_num 795) dirty exists 2023-06-27T12:45:04.273 INFO:tasks.rados.rados.0.smithi138.stdout:1663: left oid 31 (ObjNum 795 snap 196 seq_num 795) 2023-06-27T12:45:04.273 INFO:tasks.rados.rados.0.smithi138.stdout:1666: finishing write tid 1 to smithi138138488-38 2023-06-27T12:45:04.273 INFO:tasks.rados.rados.0.smithi138.stdout:1666: finishing write tid 2 to smithi138138488-38 2023-06-27T12:45:04.273 INFO:tasks.rados.rados.0.smithi138.stdout:1666: finishing write tid 3 to smithi138138488-38 2023-06-27T12:45:04.273 INFO:tasks.rados.rados.0.smithi138.stdout:1666: finishing write tid 4 to smithi138138488-38 2023-06-27T12:45:04.273 INFO:tasks.rados.rados.0.smithi138.stdout:1666: finishing write tid 5 to smithi138138488-38 2023-06-27T12:45:04.274 INFO:tasks.rados.rados.0.smithi138.stdout:1666: oid 38 updating version 0 to 68 2023-06-27T12:45:04.274 INFO:tasks.rados.rados.0.smithi138.stdout:1666: oid 38 version 68 is already newer than 68 2023-06-27T12:45:04.274 INFO:tasks.rados.rados.0.smithi138.stdout:1666: oid 38 version 68 is already newer than 68 2023-06-27T12:45:04.274 INFO:tasks.rados.rados.0.smithi138.stdout:1666: oid 38 version 68 is already newer than 68 2023-06-27T12:45:04.274 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 68 (ObjNum 798 snap 196 seq_num 798) dirty exists 2023-06-27T12:45:04.274 INFO:tasks.rados.rados.0.smithi138.stdout:1666: left oid 38 (ObjNum 798 snap 196 seq_num 798) 2023-06-27T12:45:04.274 INFO:tasks.rados.rados.0.smithi138.stdout:1668: finishing write tid 1 to smithi138138488-28 2023-06-27T12:45:04.275 INFO:tasks.rados.rados.0.smithi138.stdout:1668: finishing write tid 2 to smithi138138488-28 2023-06-27T12:45:04.275 INFO:tasks.rados.rados.0.smithi138.stdout:1668: finishing write tid 3 to smithi138138488-28 2023-06-27T12:45:04.275 INFO:tasks.rados.rados.0.smithi138.stdout:1668: finishing write tid 4 to smithi138138488-28 2023-06-27T12:45:04.275 INFO:tasks.rados.rados.0.smithi138.stdout:1668: finishing write tid 5 to smithi138138488-28 2023-06-27T12:45:04.275 INFO:tasks.rados.rados.0.smithi138.stdout:1668: oid 28 updating version 0 to 289 2023-06-27T12:45:04.275 INFO:tasks.rados.rados.0.smithi138.stdout:1668: oid 28 version 289 is already newer than 289 2023-06-27T12:45:04.275 INFO:tasks.rados.rados.0.smithi138.stdout:1668: oid 28 version 289 is already newer than 289 2023-06-27T12:45:04.275 INFO:tasks.rados.rados.0.smithi138.stdout:1668: oid 28 version 289 is already newer than 289 2023-06-27T12:45:04.276 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 289 (ObjNum 799 snap 196 seq_num 799) dirty exists 2023-06-27T12:45:04.276 INFO:tasks.rados.rados.0.smithi138.stdout:1668: left oid 28 (ObjNum 799 snap 196 seq_num 799) 2023-06-27T12:45:04.585 INFO:tasks.rados.rados.0.smithi138.stdout:1662: done (9 left) 2023-06-27T12:45:04.585 INFO:tasks.rados.rados.0.smithi138.stdout:1663: done (8 left) 2023-06-27T12:45:04.586 INFO:tasks.rados.rados.0.smithi138.stdout:1664: done (7 left) 2023-06-27T12:45:04.586 INFO:tasks.rados.rados.0.smithi138.stdout:1665: done (6 left) 2023-06-27T12:45:04.586 INFO:tasks.rados.rados.0.smithi138.stdout:1666: done (5 left) 2023-06-27T12:45:04.586 INFO:tasks.rados.rados.0.smithi138.stdout:1667: done (4 left) 2023-06-27T12:45:04.586 INFO:tasks.rados.rados.0.smithi138.stdout:1668: done (3 left) 2023-06-27T12:45:04.586 INFO:tasks.rados.rados.0.smithi138.stdout:1669: done (2 left) 2023-06-27T12:45:04.586 INFO:tasks.rados.rados.0.smithi138.stdout:1670: done (1 left) 2023-06-27T12:45:04.587 INFO:tasks.rados.rados.0.smithi138.stdout:1671: done (0 left) 2023-06-27T12:45:04.587 INFO:tasks.rados.rados.0.smithi138.stdout:1672: write (excl) oid 13 current snap is 197 2023-06-27T12:45:04.587 INFO:tasks.rados.rados.0.smithi138.stdout:1672: seq_num 800 ranges {645600=416265,1797477=711598,3219453=226147} 2023-06-27T12:45:04.592 INFO:tasks.rados.rados.0.smithi138.stdout:1672: writing smithi138138488-13 from 645600 to 1061865 tid 1 2023-06-27T12:45:04.603 INFO:tasks.rados.rados.0.smithi138.stdout:1672: writing smithi138138488-13 from 1797477 to 2509075 tid 2 2023-06-27T12:45:04.608 INFO:tasks.rados.rados.0.smithi138.stdout:1672: writing smithi138138488-13 from 3219453 to 3445600 tid 3 2023-06-27T12:45:04.609 INFO:tasks.rados.rados.0.smithi138.stdout:1673: write oid 20 current snap is 197 2023-06-27T12:45:04.609 INFO:tasks.rados.rados.0.smithi138.stdout:1673: seq_num 801 ranges {662407=491514,1855959=571609,2846876=615531} 2023-06-27T12:45:04.614 INFO:tasks.rados.rados.0.smithi138.stdout:1673: writing smithi138138488-20 from 662407 to 1153921 tid 1 2023-06-27T12:45:04.621 INFO:tasks.rados.rados.0.smithi138.stdout:1673: writing smithi138138488-20 from 1855959 to 2427568 tid 2 2023-06-27T12:45:04.627 INFO:tasks.rados.rados.0.smithi138.stdout:1673: writing smithi138138488-20 from 2846876 to 3462407 tid 3 2023-06-27T12:45:04.627 INFO:tasks.rados.rados.0.smithi138.stdout:1674: write (excl) oid 18 current snap is 197 2023-06-27T12:45:04.627 INFO:tasks.rados.rados.0.smithi138.stdout:1674: seq_num 802 ranges {679214=566763,1798088=715267,3157946=321268} 2023-06-27T12:45:04.634 INFO:tasks.rados.rados.0.smithi138.stdout:1674: writing smithi138138488-18 from 679214 to 1245977 tid 1 2023-06-27T12:45:04.641 INFO:tasks.rados.rados.0.smithi138.stdout:1674: writing smithi138138488-18 from 1798088 to 2513355 tid 2 2023-06-27T12:45:04.647 INFO:tasks.rados.rados.0.smithi138.stdout:1674: writing smithi138138488-18 from 3157946 to 3479214 tid 3 2023-06-27T12:45:04.647 INFO:tasks.rados.rados.0.smithi138.stdout:1675: write oid 36 current snap is 197 2023-06-27T12:45:04.647 INFO:tasks.rados.rados.0.smithi138.stdout:1675: seq_num 803 ranges {696021=642012,1856570=575278,3185369=310652} 2023-06-27T12:45:04.654 INFO:tasks.rados.rados.0.smithi138.stdout:1675: writing smithi138138488-36 from 696021 to 1338033 tid 1 2023-06-27T12:45:04.660 INFO:tasks.rados.rados.0.smithi138.stdout:1675: writing smithi138138488-36 from 1856570 to 2431848 tid 2 2023-06-27T12:45:04.667 INFO:tasks.rados.rados.0.smithi138.stdout:1675: writing smithi138138488-36 from 3185369 to 3496021 tid 3 2023-06-27T12:45:04.667 INFO:tasks.rados.rados.0.smithi138.stdout:1676: write oid 7 current snap is 197 2023-06-27T12:45:04.667 INFO:tasks.rados.rados.0.smithi138.stdout:1674: finishing write tid 1 to smithi138138488-18 2023-06-27T12:45:04.667 INFO:tasks.rados.rados.0.smithi138.stdout:1672: finishing write tid 1 to smithi138138488-13 2023-06-27T12:45:04.667 INFO:tasks.rados.rados.0.smithi138.stdout:1676: seq_num 804 ranges {712828=717261,1915052=718936,3212792=300036} 2023-06-27T12:45:04.675 INFO:tasks.rados.rados.0.smithi138.stdout:1676: writing smithi138138488-7 from 712828 to 1430089 tid 1 2023-06-27T12:45:04.683 INFO:tasks.rados.rados.0.smithi138.stdout:1676: writing smithi138138488-7 from 1915052 to 2633988 tid 2 2023-06-27T12:45:04.689 INFO:tasks.rados.rados.0.smithi138.stdout:1676: writing smithi138138488-7 from 3212792 to 3512828 tid 3 2023-06-27T12:45:04.689 INFO:tasks.rados.rados.0.smithi138.stdout:1677: read oid 4 snap -1 2023-06-27T12:45:04.689 INFO:tasks.rados.rados.0.smithi138.stdout:1677: expect (ObjNum 682 snap 167 seq_num 682) 2023-06-27T12:45:04.689 INFO:tasks.rados.rados.0.smithi138.stdout:1678: read oid 39 snap -1 2023-06-27T12:45:04.689 INFO:tasks.rados.rados.0.smithi138.stdout:1678: expect (ObjNum 769 snap 189 seq_num 769) 2023-06-27T12:45:04.690 INFO:tasks.rados.rados.0.smithi138.stdout:1679: snap_remove snap 193 2023-06-27T12:45:05.017 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~270s 2023-06-27T12:45:05.100 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:05.227 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:05.221+0000 7facc18e5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:05.229 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:05.225+0000 7facc18e5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:05.275 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:05.278 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:05.279 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:05.594 INFO:tasks.rados.rados.0.smithi138.stdout:1679: done (7 left) 2023-06-27T12:45:05.594 INFO:tasks.rados.rados.0.smithi138.stdout:1680: read oid 43 snap -1 2023-06-27T12:45:05.594 INFO:tasks.rados.rados.0.smithi138.stdout:1680: expect (ObjNum 779 snap 191 seq_num 779) 2023-06-27T12:45:05.595 INFO:tasks.rados.rados.0.smithi138.stdout:1681: delete oid 47 current snap is 197 2023-06-27T12:45:05.595 INFO:tasks.rados.rados.0.smithi138.stdout:1673: finishing write tid 1 to smithi138138488-20 2023-06-27T12:45:05.595 INFO:tasks.rados.rados.0.smithi138.stdout:1673: finishing write tid 2 to smithi138138488-20 2023-06-27T12:45:05.595 INFO:tasks.rados.rados.0.smithi138.stdout:1674: finishing write tid 2 to smithi138138488-18 2023-06-27T12:45:05.595 INFO:tasks.rados.rados.0.smithi138.stdout:1672: finishing write tid 2 to smithi138138488-13 2023-06-27T12:45:05.595 INFO:tasks.rados.rados.0.smithi138.stdout:1672: finishing write tid 3 to smithi138138488-13 2023-06-27T12:45:05.595 INFO:tasks.rados.rados.0.smithi138.stdout:1672: finishing write tid 4 to smithi138138488-13 2023-06-27T12:45:05.595 INFO:tasks.rados.rados.0.smithi138.stdout:1672: finishing write tid 5 to smithi138138488-13 2023-06-27T12:45:05.596 INFO:tasks.rados.rados.0.smithi138.stdout:1672: oid 13 updating version 0 to 521 2023-06-27T12:45:05.596 INFO:tasks.rados.rados.0.smithi138.stdout:1672: oid 13 version 521 is already newer than 521 2023-06-27T12:45:05.596 INFO:tasks.rados.rados.0.smithi138.stdout:1672: oid 13 version 521 is already newer than 521 2023-06-27T12:45:05.596 INFO:tasks.rados.rados.0.smithi138.stdout:1672: oid 13 version 521 is already newer than 517 2023-06-27T12:45:05.596 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 13 v 521 (ObjNum 800 snap 197 seq_num 800) dirty exists 2023-06-27T12:45:05.596 INFO:tasks.rados.rados.0.smithi138.stdout:1672: left oid 13 (ObjNum 800 snap 197 seq_num 800) 2023-06-27T12:45:05.597 INFO:tasks.rados.rados.0.smithi138.stdout:1674: finishing write tid 3 to smithi138138488-18 2023-06-27T12:45:05.597 INFO:tasks.rados.rados.0.smithi138.stdout:1676: finishing write tid 1 to smithi138138488-7 2023-06-27T12:45:05.597 INFO:tasks.rados.rados.0.smithi138.stdout:1674: finishing write tid 4 to smithi138138488-18 2023-06-27T12:45:05.597 INFO:tasks.rados.rados.0.smithi138.stdout:1674: finishing write tid 5 to smithi138138488-18 2023-06-27T12:45:05.597 INFO:tasks.rados.rados.0.smithi138.stdout:1674: oid 18 updating version 0 to 347 2023-06-27T12:45:05.597 INFO:tasks.rados.rados.0.smithi138.stdout:1674: oid 18 version 347 is already newer than 347 2023-06-27T12:45:05.597 INFO:tasks.rados.rados.0.smithi138.stdout:1674: oid 18 version 347 is already newer than 344 2023-06-27T12:45:05.598 INFO:tasks.rados.rados.0.smithi138.stdout:1674: oid 18 version 347 is already newer than 346 2023-06-27T12:45:05.598 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 347 (ObjNum 802 snap 197 seq_num 802) dirty exists 2023-06-27T12:45:05.598 INFO:tasks.rados.rados.0.smithi138.stdout:1674: left oid 18 (ObjNum 802 snap 197 seq_num 802) 2023-06-27T12:45:05.598 INFO:tasks.rados.rados.0.smithi138.stdout:1676: finishing write tid 2 to smithi138138488-7 2023-06-27T12:45:05.598 INFO:tasks.rados.rados.0.smithi138.stdout:1673: finishing write tid 3 to smithi138138488-20 2023-06-27T12:45:05.598 INFO:tasks.rados.rados.0.smithi138.stdout:1673: finishing write tid 4 to smithi138138488-20 2023-06-27T12:45:05.598 INFO:tasks.rados.rados.0.smithi138.stdout:1673: finishing write tid 5 to smithi138138488-20 2023-06-27T12:45:05.598 INFO:tasks.rados.rados.0.smithi138.stdout:1673: oid 20 updating version 0 to 310 2023-06-27T12:45:05.599 INFO:tasks.rados.rados.0.smithi138.stdout:1673: oid 20 version 310 is already newer than 310 2023-06-27T12:45:05.599 INFO:tasks.rados.rados.0.smithi138.stdout:1673: oid 20 version 310 is already newer than 310 2023-06-27T12:45:05.599 INFO:tasks.rados.rados.0.smithi138.stdout:1673: oid 20 version 310 is already newer than 310 2023-06-27T12:45:05.599 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 310 (ObjNum 801 snap 197 seq_num 801) dirty exists 2023-06-27T12:45:05.599 INFO:tasks.rados.rados.0.smithi138.stdout:1673: left oid 20 (ObjNum 801 snap 197 seq_num 801) 2023-06-27T12:45:05.599 INFO:tasks.rados.rados.0.smithi138.stdout:1676: finishing write tid 3 to smithi138138488-7 2023-06-27T12:45:05.599 INFO:tasks.rados.rados.0.smithi138.stdout:1676: finishing write tid 4 to smithi138138488-7 2023-06-27T12:45:05.600 INFO:tasks.rados.rados.0.smithi138.stdout:1676: finishing write tid 5 to smithi138138488-7 2023-06-27T12:45:05.600 INFO:tasks.rados.rados.0.smithi138.stdout:1676: oid 7 updating version 0 to 294 2023-06-27T12:45:05.600 INFO:tasks.rados.rados.0.smithi138.stdout:1676: oid 7 version 294 is already newer than 294 2023-06-27T12:45:05.600 INFO:tasks.rados.rados.0.smithi138.stdout:1676: oid 7 version 294 is already newer than 294 2023-06-27T12:45:05.600 INFO:tasks.rados.rados.0.smithi138.stdout:1676: oid 7 version 294 is already newer than 294 2023-06-27T12:45:05.600 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 294 (ObjNum 804 snap 197 seq_num 804) dirty exists 2023-06-27T12:45:05.601 INFO:tasks.rados.rados.0.smithi138.stdout:1676: left oid 7 (ObjNum 804 snap 197 seq_num 804) 2023-06-27T12:45:05.601 INFO:tasks.rados.rados.0.smithi138.stdout:1675: finishing write tid 1 to smithi138138488-36 2023-06-27T12:45:05.601 INFO:tasks.rados.rados.0.smithi138.stdout:1675: finishing write tid 2 to smithi138138488-36 2023-06-27T12:45:05.601 INFO:tasks.rados.rados.0.smithi138.stdout:1677: expect (ObjNum 682 snap 167 seq_num 682) 2023-06-27T12:45:05.828 INFO:tasks.rados.rados.0.smithi138.stdout:1675: finishing write tid 3 to smithi138138488-36 2023-06-27T12:45:05.829 INFO:tasks.rados.rados.0.smithi138.stdout:1675: finishing write tid 4 to smithi138138488-36 2023-06-27T12:45:05.829 INFO:tasks.rados.rados.0.smithi138.stdout:1675: finishing write tid 5 to smithi138138488-36 2023-06-27T12:45:05.829 INFO:tasks.rados.rados.0.smithi138.stdout:1675: oid 36 updating version 0 to 352 2023-06-27T12:45:05.829 INFO:tasks.rados.rados.0.smithi138.stdout:1675: oid 36 version 352 is already newer than 352 2023-06-27T12:45:05.829 INFO:tasks.rados.rados.0.smithi138.stdout:1675: oid 36 version 352 is already newer than 352 2023-06-27T12:45:05.829 INFO:tasks.rados.rados.0.smithi138.stdout:1675: oid 36 version 352 is already newer than 352 2023-06-27T12:45:05.830 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 352 (ObjNum 803 snap 197 seq_num 803) dirty exists 2023-06-27T12:45:05.830 INFO:tasks.rados.rados.0.smithi138.stdout:1675: left oid 36 (ObjNum 803 snap 197 seq_num 803) 2023-06-27T12:45:05.830 INFO:tasks.rados.rados.0.smithi138.stdout:1678: expect (ObjNum 769 snap 189 seq_num 769) 2023-06-27T12:45:06.014 INFO:tasks.rados.rados.0.smithi138.stdout:1680: expect (ObjNum 779 snap 191 seq_num 779) 2023-06-27T12:45:06.221 INFO:tasks.rados.rados.0.smithi138.stdout:1672: done (8 left) 2023-06-27T12:45:06.221 INFO:tasks.rados.rados.0.smithi138.stdout:1673: done (7 left) 2023-06-27T12:45:06.221 INFO:tasks.rados.rados.0.smithi138.stdout:1674: done (6 left) 2023-06-27T12:45:06.221 INFO:tasks.rados.rados.0.smithi138.stdout:1675: done (5 left) 2023-06-27T12:45:06.221 INFO:tasks.rados.rados.0.smithi138.stdout:1676: done (4 left) 2023-06-27T12:45:06.222 INFO:tasks.rados.rados.0.smithi138.stdout:1677: done (3 left) 2023-06-27T12:45:06.222 INFO:tasks.rados.rados.0.smithi138.stdout:1678: done (2 left) 2023-06-27T12:45:06.222 INFO:tasks.rados.rados.0.smithi138.stdout:1680: done (1 left) 2023-06-27T12:45:06.222 INFO:tasks.rados.rados.0.smithi138.stdout:1681: done (0 left) 2023-06-27T12:45:06.222 INFO:tasks.rados.rados.0.smithi138.stdout:1682: read oid 42 snap -1 2023-06-27T12:45:06.222 INFO:tasks.rados.rados.0.smithi138.stdout:1682: expect (ObjNum 784 snap 192 seq_num 784) 2023-06-27T12:45:06.222 INFO:tasks.rados.rados.0.smithi138.stdout:1683: delete oid 2 current snap is 197 2023-06-27T12:45:06.246 INFO:tasks.rados.rados.0.smithi138.stdout:1682: expect (ObjNum 784 snap 192 seq_num 784) 2023-06-27T12:45:06.430 INFO:tasks.rados.rados.0.smithi138.stdout:1682: done (1 left) 2023-06-27T12:45:06.431 INFO:tasks.rados.rados.0.smithi138.stdout:1683: done (0 left) 2023-06-27T12:45:06.431 INFO:tasks.rados.rados.0.smithi138.stdout:1684: read oid 4 snap 194 2023-06-27T12:45:06.431 INFO:tasks.rados.rados.0.smithi138.stdout:1684: expect (ObjNum 682 snap 167 seq_num 682) 2023-06-27T12:45:06.431 INFO:tasks.rados.rados.0.smithi138.stdout:1685: write (excl) oid 28 current snap is 197 2023-06-27T12:45:06.431 INFO:tasks.rados.rados.0.smithi138.stdout:1685: seq_num 807 ranges {763249=659361,2090498=582616,3295061=268188} 2023-06-27T12:45:06.438 INFO:tasks.rados.rados.0.smithi138.stdout:1685: writing smithi138138488-28 from 763249 to 1422610 tid 1 2023-06-27T12:45:06.445 INFO:tasks.rados.rados.0.smithi138.stdout:1685: writing smithi138138488-28 from 2090498 to 2673114 tid 2 2023-06-27T12:45:06.450 INFO:tasks.rados.rados.0.smithi138.stdout:1685: writing smithi138138488-28 from 3295061 to 3563249 tid 3 2023-06-27T12:45:06.451 INFO:tasks.rados.rados.0.smithi138.stdout:1686: read oid 34 snap -1 2023-06-27T12:45:06.451 INFO:tasks.rados.rados.0.smithi138.stdout:1686: expect (ObjNum 724 snap 178 seq_num 724) 2023-06-27T12:45:06.451 INFO:tasks.rados.rados.0.smithi138.stdout:1684: expect (ObjNum 682 snap 167 seq_num 682) 2023-06-27T12:45:06.685 INFO:tasks.rados.rados.0.smithi138.stdout:1685: finishing write tid 1 to smithi138138488-28 2023-06-27T12:45:06.686 INFO:tasks.rados.rados.0.smithi138.stdout:1685: finishing write tid 2 to smithi138138488-28 2023-06-27T12:45:06.686 INFO:tasks.rados.rados.0.smithi138.stdout:1685: finishing write tid 3 to smithi138138488-28 2023-06-27T12:45:06.686 INFO:tasks.rados.rados.0.smithi138.stdout:1685: finishing write tid 4 to smithi138138488-28 2023-06-27T12:45:06.686 INFO:tasks.rados.rados.0.smithi138.stdout:1685: finishing write tid 5 to smithi138138488-28 2023-06-27T12:45:06.686 INFO:tasks.rados.rados.0.smithi138.stdout:1685: oid 28 updating version 0 to 303 2023-06-27T12:45:06.686 INFO:tasks.rados.rados.0.smithi138.stdout:1685: oid 28 version 303 is already newer than 303 2023-06-27T12:45:06.687 INFO:tasks.rados.rados.0.smithi138.stdout:1685: oid 28 version 303 is already newer than 303 2023-06-27T12:45:06.687 INFO:tasks.rados.rados.0.smithi138.stdout:1685: oid 28 version 303 is already newer than 299 2023-06-27T12:45:06.687 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 303 (ObjNum 807 snap 197 seq_num 807) dirty exists 2023-06-27T12:45:06.687 INFO:tasks.rados.rados.0.smithi138.stdout:1685: left oid 28 (ObjNum 807 snap 197 seq_num 807) 2023-06-27T12:45:06.687 INFO:tasks.rados.rados.0.smithi138.stdout:1684: done (2 left) 2023-06-27T12:45:06.687 INFO:tasks.rados.rados.0.smithi138.stdout:1685: done (1 left) 2023-06-27T12:45:06.687 INFO:tasks.rados.rados.0.smithi138.stdout:1687: write (excl) oid 12 current snap is 197 2023-06-27T12:45:06.687 INFO:tasks.rados.rados.0.smithi138.stdout:1687: seq_num 808 ranges {780056=734610,2148980=726274,3322484=257572} 2023-06-27T12:45:06.695 INFO:tasks.rados.rados.0.smithi138.stdout:1687: writing smithi138138488-12 from 780056 to 1514666 tid 1 2023-06-27T12:45:06.703 INFO:tasks.rados.rados.0.smithi138.stdout:1687: writing smithi138138488-12 from 2148980 to 2875254 tid 2 2023-06-27T12:45:06.707 INFO:tasks.rados.rados.0.smithi138.stdout:1687: writing smithi138138488-12 from 3322484 to 3580056 tid 3 2023-06-27T12:45:06.707 INFO:tasks.rados.rados.0.smithi138.stdout:1688: write oid 1 current snap is 197 2023-06-27T12:45:06.707 INFO:tasks.rados.rados.0.smithi138.stdout:1688: seq_num 809 ranges {796863=409859,1807462=586285,3066260=530603} 2023-06-27T12:45:06.714 INFO:tasks.rados.rados.0.smithi138.stdout:1688: writing smithi138138488-1 from 796863 to 1206722 tid 1 2023-06-27T12:45:06.721 INFO:tasks.rados.rados.0.smithi138.stdout:1688: writing smithi138138488-1 from 1807462 to 2393747 tid 2 2023-06-27T12:45:06.727 INFO:tasks.rados.rados.0.smithi138.stdout:1688: writing smithi138138488-1 from 3066260 to 3596863 tid 3 2023-06-27T12:45:06.728 INFO:tasks.rados.rados.0.smithi138.stdout:1689: snap_create 2023-06-27T12:45:06.728 INFO:tasks.rados.rados.0.smithi138.stdout:1686: expect (ObjNum 724 snap 178 seq_num 724) 2023-06-27T12:45:06.824 INFO:tasks.rados.rados.0.smithi138.stdout:1687: finishing write tid 1 to smithi138138488-12 2023-06-27T12:45:06.824 INFO:tasks.rados.rados.0.smithi138.stdout:1688: finishing write tid 1 to smithi138138488-1 2023-06-27T12:45:06.824 INFO:tasks.rados.rados.0.smithi138.stdout:1687: finishing write tid 2 to smithi138138488-12 2023-06-27T12:45:06.824 INFO:tasks.rados.rados.0.smithi138.stdout:1688: finishing write tid 2 to smithi138138488-1 2023-06-27T12:45:06.824 INFO:tasks.rados.rados.0.smithi138.stdout:1688: finishing write tid 3 to smithi138138488-1 2023-06-27T12:45:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:1687: finishing write tid 3 to smithi138138488-12 2023-06-27T12:45:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:1687: finishing write tid 4 to smithi138138488-12 2023-06-27T12:45:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:1688: finishing write tid 4 to smithi138138488-1 2023-06-27T12:45:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:1687: finishing write tid 5 to smithi138138488-12 2023-06-27T12:45:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:1687: oid 12 updating version 0 to 359 2023-06-27T12:45:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:1687: oid 12 version 359 is already newer than 359 2023-06-27T12:45:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:1687: oid 12 version 359 is already newer than 355 2023-06-27T12:45:06.825 INFO:tasks.rados.rados.0.smithi138.stdout:1687: oid 12 version 359 is already newer than 359 2023-06-27T12:45:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 12 v 359 (ObjNum 808 snap 197 seq_num 808) dirty exists 2023-06-27T12:45:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:1687: left oid 12 (ObjNum 808 snap 197 seq_num 808) 2023-06-27T12:45:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:1688: finishing write tid 5 to smithi138138488-1 2023-06-27T12:45:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:1688: oid 1 updating version 0 to 493 2023-06-27T12:45:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:1688: oid 1 version 493 is already newer than 493 2023-06-27T12:45:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:1688: oid 1 version 493 is already newer than 493 2023-06-27T12:45:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:1688: oid 1 version 493 is already newer than 493 2023-06-27T12:45:06.826 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 1 v 493 (ObjNum 809 snap 197 seq_num 809) dirty exists 2023-06-27T12:45:06.827 INFO:tasks.rados.rados.0.smithi138.stdout:1688: left oid 1 (ObjNum 809 snap 197 seq_num 809) 2023-06-27T12:45:07.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:45:07.408 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:07.402+0000 7f7506d7c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:07.411 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:07.407+0000 7f7506d7c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:07.456 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:07.459 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:07.459 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:07.600 INFO:tasks.rados.rados.0.smithi138.stdout:1686: done (3 left) 2023-06-27T12:45:07.600 INFO:tasks.rados.rados.0.smithi138.stdout:1687: done (2 left) 2023-06-27T12:45:07.600 INFO:tasks.rados.rados.0.smithi138.stdout:1688: done (1 left) 2023-06-27T12:45:07.601 INFO:tasks.rados.rados.0.smithi138.stdout:1689: done (0 left) 2023-06-27T12:45:07.601 INFO:tasks.rados.rados.0.smithi138.stdout:1690: read oid 46 snap -1 2023-06-27T12:45:07.601 INFO:tasks.rados.rados.0.smithi138.stdout:1690: expect (ObjNum 758 snap 187 seq_num 758) 2023-06-27T12:45:07.601 INFO:tasks.rados.rados.0.smithi138.stdout:1691: delete oid 41 current snap is 198 2023-06-27T12:45:07.642 INFO:tasks.rados.rados.0.smithi138.stdout:1690: expect (ObjNum 758 snap 187 seq_num 758) 2023-06-27T12:45:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:1690: done (1 left) 2023-06-27T12:45:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:1691: done (0 left) 2023-06-27T12:45:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:1692: write oid 6 current snap is 198 2023-06-27T12:45:07.772 INFO:tasks.rados.rados.0.smithi138.stdout:1692: seq_num 811 ranges {430477=560357,1408073=473601,2488400=736073} 2023-06-27T12:45:07.777 INFO:tasks.rados.rados.0.smithi138.stdout:1692: writing smithi138138488-6 from 430477 to 990834 tid 1 2023-06-27T12:45:07.782 INFO:tasks.rados.rados.0.smithi138.stdout:1692: writing smithi138138488-6 from 1408073 to 1881674 tid 2 2023-06-27T12:45:07.789 INFO:tasks.rados.rados.0.smithi138.stdout:1692: writing smithi138138488-6 from 2488400 to 3224473 tid 3 2023-06-27T12:45:07.790 INFO:tasks.rados.rados.0.smithi138.stdout:1693: write (excl) oid 14 current snap is 198 2023-06-27T12:45:07.790 INFO:tasks.rados.rados.0.smithi138.stdout:1693: seq_num 812 ranges {447284=635606,1866555=733612,3315823=331461} 2023-06-27T12:45:07.795 INFO:tasks.rados.rados.0.smithi138.stdout:1693: writing smithi138138488-14 from 447284 to 1082890 tid 1 2023-06-27T12:45:07.805 INFO:tasks.rados.rados.0.smithi138.stdout:1693: writing smithi138138488-14 from 1866555 to 2600167 tid 2 2023-06-27T12:45:07.812 INFO:tasks.rados.rados.0.smithi138.stdout:1693: writing smithi138138488-14 from 3315823 to 3647284 tid 3 2023-06-27T12:45:07.812 INFO:tasks.rados.rados.0.smithi138.stdout:1694: write (excl) oid 26 current snap is 198 2023-06-27T12:45:07.812 INFO:tasks.rados.rados.0.smithi138.stdout:1694: seq_num 813 ranges {464091=710855,1925037=477270,2943246=474970} 2023-06-27T12:45:07.820 INFO:tasks.rados.rados.0.smithi138.stdout:1694: writing smithi138138488-26 from 464091 to 1174946 tid 1 2023-06-27T12:45:07.827 INFO:tasks.rados.rados.0.smithi138.stdout:1694: writing smithi138138488-26 from 1925037 to 2402307 tid 2 2023-06-27T12:45:07.833 INFO:tasks.rados.rados.0.smithi138.stdout:1694: writing smithi138138488-26 from 2943246 to 3418216 tid 3 2023-06-27T12:45:07.834 INFO:tasks.rados.rados.0.smithi138.stdout:1695: write (excl) oid 38 current snap is 198 2023-06-27T12:45:07.834 INFO:tasks.rados.rados.0.smithi138.stdout:1695: seq_num 814 ranges {480898=502457,1699872=737281,3087022=593876} 2023-06-27T12:45:07.839 INFO:tasks.rados.rados.0.smithi138.stdout:1695: writing smithi138138488-38 from 480898 to 983355 tid 1 2023-06-27T12:45:07.847 INFO:tasks.rados.rados.0.smithi138.stdout:1695: writing smithi138138488-38 from 1699872 to 2437153 tid 2 2023-06-27T12:45:07.854 INFO:tasks.rados.rados.0.smithi138.stdout:1695: writing smithi138138488-38 from 3087022 to 3680898 tid 3 2023-06-27T12:45:07.855 INFO:tasks.rados.rados.0.smithi138.stdout:1696: read oid 35 snap 195 2023-06-27T12:45:07.855 INFO:tasks.rados.rados.0.smithi138.stdout:1696: expect (ObjNum 788 snap 192 seq_num 788) 2023-06-27T12:45:07.855 INFO:tasks.rados.rados.0.smithi138.stdout:1697: delete oid 48 current snap is 198 2023-06-27T12:45:07.855 INFO:tasks.rados.rados.0.smithi138.stdout:1692: finishing write tid 1 to smithi138138488-6 2023-06-27T12:45:07.855 INFO:tasks.rados.rados.0.smithi138.stdout:1692: finishing write tid 2 to smithi138138488-6 2023-06-27T12:45:07.875 INFO:tasks.rados.rados.0.smithi138.stdout:1692: finishing write tid 3 to smithi138138488-6 2023-06-27T12:45:07.877 INFO:tasks.rados.rados.0.smithi138.stdout:1696: expect (ObjNum 788 snap 192 seq_num 788) 2023-06-27T12:45:08.058 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:45:08.059 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-06-27T12:45:08.171 INFO:tasks.rados.rados.0.smithi138.stdout:1692: finishing write tid 4 to smithi138138488-6 2023-06-27T12:45:08.171 INFO:tasks.rados.rados.0.smithi138.stdout:1694: finishing write tid 1 to smithi138138488-26 2023-06-27T12:45:08.171 INFO:tasks.rados.rados.0.smithi138.stdout:1693: finishing write tid 1 to smithi138138488-14 2023-06-27T12:45:08.172 INFO:tasks.rados.rados.0.smithi138.stdout:1695: finishing write tid 1 to smithi138138488-38 2023-06-27T12:45:08.172 INFO:tasks.rados.rados.0.smithi138.stdout:1692: finishing write tid 5 to smithi138138488-6 2023-06-27T12:45:08.172 INFO:tasks.rados.rados.0.smithi138.stdout:1692: oid 6 updating version 0 to 238 2023-06-27T12:45:08.172 INFO:tasks.rados.rados.0.smithi138.stdout:1692: oid 6 updating version 238 to 239 2023-06-27T12:45:08.172 INFO:tasks.rados.rados.0.smithi138.stdout:1692: oid 6 version 239 is already newer than 239 2023-06-27T12:45:08.172 INFO:tasks.rados.rados.0.smithi138.stdout:1692: oid 6 version 239 is already newer than 239 2023-06-27T12:45:08.172 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 239 (ObjNum 811 snap 198 seq_num 811) dirty exists 2023-06-27T12:45:08.173 INFO:tasks.rados.rados.0.smithi138.stdout:1692: left oid 6 (ObjNum 811 snap 198 seq_num 811) 2023-06-27T12:45:08.173 INFO:tasks.rados.rados.0.smithi138.stdout:1694: finishing write tid 2 to smithi138138488-26 2023-06-27T12:45:08.173 INFO:tasks.rados.rados.0.smithi138.stdout:1694: finishing write tid 3 to smithi138138488-26 2023-06-27T12:45:08.173 INFO:tasks.rados.rados.0.smithi138.stdout:1694: finishing write tid 4 to smithi138138488-26 2023-06-27T12:45:08.173 INFO:tasks.rados.rados.0.smithi138.stdout:1695: finishing write tid 2 to smithi138138488-38 2023-06-27T12:45:08.173 INFO:tasks.rados.rados.0.smithi138.stdout:1695: finishing write tid 3 to smithi138138488-38 2023-06-27T12:45:08.173 INFO:tasks.rados.rados.0.smithi138.stdout:1695: finishing write tid 4 to smithi138138488-38 2023-06-27T12:45:08.173 INFO:tasks.rados.rados.0.smithi138.stdout:1692: done (5 left) 2023-06-27T12:45:08.174 INFO:tasks.rados.rados.0.smithi138.stdout:1696: done (4 left) 2023-06-27T12:45:08.174 INFO:tasks.rados.rados.0.smithi138.stdout:1697: done (3 left) 2023-06-27T12:45:08.174 INFO:tasks.rados.rados.0.smithi138.stdout:1698: snap_remove snap 194 2023-06-27T12:45:08.191 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:08.182+0000 7f8eaf186700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:08.194 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:08.190+0000 7f8eaf186700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:08.617 INFO:tasks.rados.rados.0.smithi138.stdout:1698: done (3 left) 2023-06-27T12:45:08.617 INFO:tasks.rados.rados.0.smithi138.stdout:1699: write oid 10 current snap is 198 2023-06-27T12:45:08.617 INFO:tasks.rados.rados.0.smithi138.stdout:1694: finishing write tid 5 to smithi138138488-26 2023-06-27T12:45:08.618 INFO:tasks.rados.rados.0.smithi138.stdout:1694: oid 26 updating version 0 to 289 2023-06-27T12:45:08.618 INFO:tasks.rados.rados.0.smithi138.stdout:1694: oid 26 version 289 is already newer than 285 2023-06-27T12:45:08.618 INFO:tasks.rados.rados.0.smithi138.stdout:1694: oid 26 version 289 is already newer than 289 2023-06-27T12:45:08.619 INFO:tasks.rados.rados.0.smithi138.stdout:1694: oid 26 version 289 is already newer than 289 2023-06-27T12:45:08.619 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 289 (ObjNum 813 snap 198 seq_num 813) dirty exists 2023-06-27T12:45:08.619 INFO:tasks.rados.rados.0.smithi138.stdout:1694: left oid 26 (ObjNum 813 snap 198 seq_num 813) 2023-06-27T12:45:08.619 INFO:tasks.rados.rados.0.smithi138.stdout:1695: finishing write tid 5 to smithi138138488-38 2023-06-27T12:45:08.620 INFO:tasks.rados.rados.0.smithi138.stdout:1695: oid 38 updating version 0 to 75 2023-06-27T12:45:08.620 INFO:tasks.rados.rados.0.smithi138.stdout:1695: oid 38 version 75 is already newer than 75 2023-06-27T12:45:08.620 INFO:tasks.rados.rados.0.smithi138.stdout:1695: oid 38 version 75 is already newer than 71 2023-06-27T12:45:08.621 INFO:tasks.rados.rados.0.smithi138.stdout:1695: oid 38 version 75 is already newer than 75 2023-06-27T12:45:08.621 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 75 (ObjNum 814 snap 198 seq_num 814) dirty exists 2023-06-27T12:45:08.621 INFO:tasks.rados.rados.0.smithi138.stdout:1695: left oid 38 (ObjNum 814 snap 198 seq_num 814) 2023-06-27T12:45:08.621 INFO:tasks.rados.rados.0.smithi138.stdout:1693: finishing write tid 2 to smithi138138488-14 2023-06-27T12:45:08.622 INFO:tasks.rados.rados.0.smithi138.stdout:1693: finishing write tid 3 to smithi138138488-14 2023-06-27T12:45:08.622 INFO:tasks.rados.rados.0.smithi138.stdout:1693: finishing write tid 4 to smithi138138488-14 2023-06-27T12:45:08.622 INFO:tasks.rados.rados.0.smithi138.stdout:1693: finishing write tid 5 to smithi138138488-14 2023-06-27T12:45:08.623 INFO:tasks.rados.rados.0.smithi138.stdout:1693: oid 14 updating version 0 to 498 2023-06-27T12:45:08.623 INFO:tasks.rados.rados.0.smithi138.stdout:1693: oid 14 version 498 is already newer than 498 2023-06-27T12:45:08.623 INFO:tasks.rados.rados.0.smithi138.stdout:1693: oid 14 version 498 is already newer than 498 2023-06-27T12:45:08.623 INFO:tasks.rados.rados.0.smithi138.stdout:1693: oid 14 version 498 is already newer than 494 2023-06-27T12:45:08.623 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 498 (ObjNum 812 snap 198 seq_num 812) dirty exists 2023-06-27T12:45:08.624 INFO:tasks.rados.rados.0.smithi138.stdout:1693: left oid 14 (ObjNum 812 snap 198 seq_num 812) 2023-06-27T12:45:08.624 INFO:tasks.rados.rados.0.smithi138.stdout:1699: seq_num 816 ranges {514512=652955,1700483=740950,3025515=688997} 2023-06-27T12:45:08.624 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 1 (10000) 2023-06-27T12:45:08.625 INFO:tasks.rados.rados.0.smithi138.stdout:1699: writing smithi138138488-10 from 514512 to 1167467 tid 1 2023-06-27T12:45:08.635 INFO:tasks.rados.rados.0.smithi138.stdout:1699: writing smithi138138488-10 from 1700483 to 2441433 tid 2 2023-06-27T12:45:08.638 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-06-27T12:45:08.644 INFO:tasks.rados.rados.0.smithi138.stdout:1699: writing smithi138138488-10 from 3025515 to 3714512 tid 3 2023-06-27T12:45:08.644 INFO:tasks.rados.rados.0.smithi138.stdout:1693: done (3 left) 2023-06-27T12:45:08.644 INFO:tasks.rados.rados.0.smithi138.stdout:1694: done (2 left) 2023-06-27T12:45:08.644 INFO:tasks.rados.rados.0.smithi138.stdout:1695: done (1 left) 2023-06-27T12:45:08.644 INFO:tasks.rados.rados.0.smithi138.stdout:1700: read oid 37 snap -1 2023-06-27T12:45:08.645 INFO:tasks.rados.rados.0.smithi138.stdout:1700: expect (ObjNum 771 snap 189 seq_num 771) 2023-06-27T12:45:08.645 INFO:tasks.rados.rados.0.smithi138.stdout:1701: write oid 8 current snap is 198 2023-06-27T12:45:08.645 INFO:tasks.rados.rados.0.smithi138.stdout:1701: seq_num 817 ranges {531319=728204,1758965=484608,2652938=520058} 2023-06-27T12:45:08.651 INFO:tasks.rados.rados.0.smithi138.stdout:1701: writing smithi138138488-8 from 531319 to 1259523 tid 1 2023-06-27T12:45:08.658 INFO:tasks.rados.rados.0.smithi138.stdout:1701: writing smithi138138488-8 from 1758965 to 2243573 tid 2 2023-06-27T12:45:08.665 INFO:tasks.rados.rados.0.smithi138.stdout:1701: writing smithi138138488-8 from 2652938 to 3172996 tid 3 2023-06-27T12:45:08.665 INFO:tasks.rados.rados.0.smithi138.stdout:1702: read oid 11 snap -1 2023-06-27T12:45:08.666 INFO:tasks.rados.rados.0.smithi138.stdout:1702: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:45:08.666 INFO:tasks.rados.rados.0.smithi138.stdout:1703: read oid 28 snap -1 2023-06-27T12:45:08.666 INFO:tasks.rados.rados.0.smithi138.stdout:1703: expect (ObjNum 807 snap 197 seq_num 807) 2023-06-27T12:45:08.667 INFO:tasks.rados.rados.0.smithi138.stdout:1704: snap_create 2023-06-27T12:45:08.785 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:08.775+0000 7ff109dbc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:08.789 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:08.785+0000 7ff109dbc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:08.789 INFO:tasks.rados.rados.0.smithi138.stdout:1700: expect (ObjNum 771 snap 189 seq_num 771) 2023-06-27T12:45:08.948 INFO:tasks.rados.rados.0.smithi138.stdout:1702: expect (ObjNum 488 snap 113 seq_num 488) 2023-06-27T12:45:09.033 INFO:tasks.rados.rados.0.smithi138.stdout:1701: finishing write tid 1 to smithi138138488-8 2023-06-27T12:45:09.034 INFO:tasks.rados.rados.0.smithi138.stdout:1701: finishing write tid 2 to smithi138138488-8 2023-06-27T12:45:09.034 INFO:tasks.rados.rados.0.smithi138.stdout:1701: finishing write tid 3 to smithi138138488-8 2023-06-27T12:45:09.034 INFO:tasks.rados.rados.0.smithi138.stdout:1699: finishing write tid 1 to smithi138138488-10 2023-06-27T12:45:09.034 INFO:tasks.rados.rados.0.smithi138.stdout:1701: finishing write tid 4 to smithi138138488-8 2023-06-27T12:45:09.034 INFO:tasks.rados.rados.0.smithi138.stdout:1701: finishing write tid 5 to smithi138138488-8 2023-06-27T12:45:09.034 INFO:tasks.rados.rados.0.smithi138.stdout:1701: oid 8 updating version 0 to 115 2023-06-27T12:45:09.034 INFO:tasks.rados.rados.0.smithi138.stdout:1701: oid 8 version 115 is already newer than 115 2023-06-27T12:45:09.034 INFO:tasks.rados.rados.0.smithi138.stdout:1701: oid 8 version 115 is already newer than 115 2023-06-27T12:45:09.035 INFO:tasks.rados.rados.0.smithi138.stdout:1701: oid 8 version 115 is already newer than 115 2023-06-27T12:45:09.035 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 8 v 115 (ObjNum 817 snap 198 seq_num 817) dirty exists 2023-06-27T12:45:09.035 INFO:tasks.rados.rados.0.smithi138.stdout:1701: left oid 8 (ObjNum 817 snap 198 seq_num 817) 2023-06-27T12:45:09.036 INFO:tasks.rados.rados.0.smithi138.stdout:1699: finishing write tid 2 to smithi138138488-10 2023-06-27T12:45:09.036 INFO:tasks.rados.rados.0.smithi138.stdout:1699: finishing write tid 3 to smithi138138488-10 2023-06-27T12:45:09.036 INFO:tasks.rados.rados.0.smithi138.stdout:1699: finishing write tid 4 to smithi138138488-10 2023-06-27T12:45:09.036 INFO:tasks.rados.rados.0.smithi138.stdout:1699: finishing write tid 5 to smithi138138488-10 2023-06-27T12:45:09.037 INFO:tasks.rados.rados.0.smithi138.stdout:1699: oid 10 updating version 0 to 312 2023-06-27T12:45:09.037 INFO:tasks.rados.rados.0.smithi138.stdout:1699: oid 10 version 312 is already newer than 312 2023-06-27T12:45:09.037 INFO:tasks.rados.rados.0.smithi138.stdout:1699: oid 10 version 312 is already newer than 312 2023-06-27T12:45:09.037 INFO:tasks.rados.rados.0.smithi138.stdout:1699: oid 10 version 312 is already newer than 312 2023-06-27T12:45:09.037 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 10 v 312 (ObjNum 816 snap 198 seq_num 816) dirty exists 2023-06-27T12:45:09.037 INFO:tasks.rados.rados.0.smithi138.stdout:1699: left oid 10 (ObjNum 816 snap 198 seq_num 816) 2023-06-27T12:45:09.037 INFO:tasks.rados.rados.0.smithi138.stdout:1703: expect (ObjNum 807 snap 197 seq_num 807) 2023-06-27T12:45:09.461 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:09.587 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:09.582+0000 7f5592d23700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:09.590 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:09.586+0000 7f5592d23700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:09.615 INFO:tasks.rados.rados.0.smithi138.stdout:1699: done (5 left) 2023-06-27T12:45:09.615 INFO:tasks.rados.rados.0.smithi138.stdout:1700: done (4 left) 2023-06-27T12:45:09.615 INFO:tasks.rados.rados.0.smithi138.stdout:1701: done (3 left) 2023-06-27T12:45:09.615 INFO:tasks.rados.rados.0.smithi138.stdout:1702: done (2 left) 2023-06-27T12:45:09.615 INFO:tasks.rados.rados.0.smithi138.stdout:1703: done (1 left) 2023-06-27T12:45:09.616 INFO:tasks.rados.rados.0.smithi138.stdout:1704: done (0 left) 2023-06-27T12:45:09.616 INFO:tasks.rados.rados.0.smithi138.stdout:1705: read oid 25 snap -1 2023-06-27T12:45:09.616 INFO:tasks.rados.rados.0.smithi138.stdout:1705: expect (ObjNum 790 snap 194 seq_num 790) 2023-06-27T12:45:09.616 INFO:tasks.rados.rados.0.smithi138.stdout:1706: read oid 32 snap 195 2023-06-27T12:45:09.617 INFO:tasks.rados.rados.0.smithi138.stdout:1706: expect deleted 2023-06-27T12:45:09.617 INFO:tasks.rados.rados.0.smithi138.stdout:1707: snap_remove snap 170 2023-06-27T12:45:09.617 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.1 to 1 (10000) 2023-06-27T12:45:09.630 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-06-27T12:45:09.654 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:09.658 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:09.659 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:09.766 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:09.756+0000 7f230b2b5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:09.768 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:09.764+0000 7f230b2b5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:10.521 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~275s 2023-06-27T12:45:10.619 INFO:tasks.rados.rados.0.smithi138.stdout:1707: done (2 left) 2023-06-27T12:45:10.620 INFO:tasks.rados.rados.0.smithi138.stdout:1708: read oid 47 snap 197 2023-06-27T12:45:10.620 INFO:tasks.rados.rados.0.smithi138.stdout:1708: expect deleted 2023-06-27T12:45:10.620 INFO:tasks.rados.rados.0.smithi138.stdout:1709: delete oid 13 current snap is 199 2023-06-27T12:45:10.620 INFO:tasks.rados.rados.0.smithi138.stdout:1705: expect (ObjNum 790 snap 194 seq_num 790) 2023-06-27T12:45:10.622 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.2 to 1 (10000) 2023-06-27T12:45:10.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-06-27T12:45:10.783 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:10.773+0000 7f6e16d6a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:10.786 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:10.782+0000 7f6e16d6a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:10.843 INFO:tasks.rados.rados.0.smithi138.stdout:1705: done (3 left) 2023-06-27T12:45:10.843 INFO:tasks.rados.rados.0.smithi138.stdout:1706: done (2 left) 2023-06-27T12:45:10.843 INFO:tasks.rados.rados.0.smithi138.stdout:1708: done (1 left) 2023-06-27T12:45:10.843 INFO:tasks.rados.rados.0.smithi138.stdout:1709: done (0 left) 2023-06-27T12:45:10.844 INFO:tasks.rados.rados.0.smithi138.stdout:1710: write oid 9 current snap is 199 2023-06-27T12:45:10.844 INFO:tasks.rados.rados.0.smithi138.stdout:1710: seq_num 819 ranges {564933=478702,1759576=488277,2991431=658955} 2023-06-27T12:45:10.848 INFO:tasks.rados.rados.0.smithi138.stdout:1710: writing smithi138138488-9 from 564933 to 1043635 tid 1 2023-06-27T12:45:10.857 INFO:tasks.rados.rados.0.smithi138.stdout:1710: writing smithi138138488-9 from 1759576 to 2247853 tid 2 2023-06-27T12:45:10.865 INFO:tasks.rados.rados.0.smithi138.stdout:1710: writing smithi138138488-9 from 2991431 to 3650386 tid 3 2023-06-27T12:45:10.865 INFO:tasks.rados.rados.0.smithi138.stdout:1711: read oid 8 snap 196 2023-06-27T12:45:10.866 INFO:tasks.rados.rados.0.smithi138.stdout:1711: expect (ObjNum 664 snap 164 seq_num 664) 2023-06-27T12:45:10.866 INFO:tasks.rados.rados.0.smithi138.stdout:1712: write oid 28 current snap is 199 2023-06-27T12:45:10.866 INFO:tasks.rados.rados.0.smithi138.stdout:1712: seq_num 820 ranges {581740=553951,1818058=748288,3018854=470227} 2023-06-27T12:45:10.871 INFO:tasks.rados.rados.0.smithi138.stdout:1712: writing smithi138138488-28 from 581740 to 1135691 tid 1 2023-06-27T12:45:10.879 INFO:tasks.rados.rados.0.smithi138.stdout:1712: writing smithi138138488-28 from 1818058 to 2566346 tid 2 2023-06-27T12:45:10.886 INFO:tasks.rados.rados.0.smithi138.stdout:1712: writing smithi138138488-28 from 3018854 to 3489081 tid 3 2023-06-27T12:45:10.887 INFO:tasks.rados.rados.0.smithi138.stdout:1713: write oid 18 current snap is 199 2023-06-27T12:45:10.887 INFO:tasks.rados.rados.0.smithi138.stdout:1713: seq_num 821 ranges {598547=629200,1876540=491946,3046277=681499} 2023-06-27T12:45:10.895 INFO:tasks.rados.rados.0.smithi138.stdout:1713: writing smithi138138488-18 from 598547 to 1227747 tid 1 2023-06-27T12:45:10.902 INFO:tasks.rados.rados.0.smithi138.stdout:1713: writing smithi138138488-18 from 1876540 to 2368486 tid 2 2023-06-27T12:45:10.910 INFO:tasks.rados.rados.0.smithi138.stdout:1713: writing smithi138138488-18 from 3046277 to 3727776 tid 3 2023-06-27T12:45:10.910 INFO:tasks.rados.rados.0.smithi138.stdout:1714: snap_remove snap 195 2023-06-27T12:45:11.623 INFO:tasks.rados.rados.0.smithi138.stdout:1714: done (4 left) 2023-06-27T12:45:11.624 INFO:tasks.rados.rados.0.smithi138.stdout:1715: read oid 42 snap -1 2023-06-27T12:45:11.624 INFO:tasks.rados.rados.0.smithi138.stdout:1715: expect (ObjNum 784 snap 192 seq_num 784) 2023-06-27T12:45:11.624 INFO:tasks.rados.rados.0.smithi138.stdout:1716: snap_create 2023-06-27T12:45:11.624 INFO:tasks.rados.rados.0.smithi138.stdout:1711: expect (ObjNum 664 snap 164 seq_num 664) 2023-06-27T12:45:11.627 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.3 to 1 (10000) 2023-06-27T12:45:11.640 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-06-27T12:45:11.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:45:11.661 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:45:11.782 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:11.773+0000 7f4aba586700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:11.785 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:11.781+0000 7f4aba586700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:11.822 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:11.817+0000 7f6d95016700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:11.825 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:11.821+0000 7f6d95016700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:11.831 INFO:tasks.rados.rados.0.smithi138.stdout:1710: finishing write tid 1 to smithi138138488-9 2023-06-27T12:45:11.831 INFO:tasks.rados.rados.0.smithi138.stdout:1710: finishing write tid 2 to smithi138138488-9 2023-06-27T12:45:11.831 INFO:tasks.rados.rados.0.smithi138.stdout:1710: finishing write tid 3 to smithi138138488-9 2023-06-27T12:45:11.831 INFO:tasks.rados.rados.0.smithi138.stdout:1710: finishing write tid 4 to smithi138138488-9 2023-06-27T12:45:11.832 INFO:tasks.rados.rados.0.smithi138.stdout:1710: finishing write tid 5 to smithi138138488-9 2023-06-27T12:45:11.832 INFO:tasks.rados.rados.0.smithi138.stdout:1710: oid 9 updating version 0 to 507 2023-06-27T12:45:11.832 INFO:tasks.rados.rados.0.smithi138.stdout:1710: oid 9 version 507 is already newer than 507 2023-06-27T12:45:11.832 INFO:tasks.rados.rados.0.smithi138.stdout:1710: oid 9 version 507 is already newer than 507 2023-06-27T12:45:11.832 INFO:tasks.rados.rados.0.smithi138.stdout:1710: oid 9 version 507 is already newer than 507 2023-06-27T12:45:11.832 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 507 (ObjNum 819 snap 199 seq_num 819) dirty exists 2023-06-27T12:45:11.832 INFO:tasks.rados.rados.0.smithi138.stdout:1710: left oid 9 (ObjNum 819 snap 199 seq_num 819) 2023-06-27T12:45:11.833 INFO:tasks.rados.rados.0.smithi138.stdout:1712: finishing write tid 1 to smithi138138488-28 2023-06-27T12:45:11.833 INFO:tasks.rados.rados.0.smithi138.stdout:1713: finishing write tid 1 to smithi138138488-18 2023-06-27T12:45:11.833 INFO:tasks.rados.rados.0.smithi138.stdout:1713: finishing write tid 2 to smithi138138488-18 2023-06-27T12:45:11.833 INFO:tasks.rados.rados.0.smithi138.stdout:1712: finishing write tid 2 to smithi138138488-28 2023-06-27T12:45:11.833 INFO:tasks.rados.rados.0.smithi138.stdout:1712: finishing write tid 3 to smithi138138488-28 2023-06-27T12:45:11.833 INFO:tasks.rados.rados.0.smithi138.stdout:1713: finishing write tid 3 to smithi138138488-18 2023-06-27T12:45:11.833 INFO:tasks.rados.rados.0.smithi138.stdout:1713: finishing write tid 4 to smithi138138488-18 2023-06-27T12:45:11.833 INFO:tasks.rados.rados.0.smithi138.stdout:1712: finishing write tid 4 to smithi138138488-28 2023-06-27T12:45:11.834 INFO:tasks.rados.rados.0.smithi138.stdout:1713: finishing write tid 5 to smithi138138488-18 2023-06-27T12:45:11.834 INFO:tasks.rados.rados.0.smithi138.stdout:1713: oid 18 updating version 0 to 372 2023-06-27T12:45:11.834 INFO:tasks.rados.rados.0.smithi138.stdout:1713: oid 18 version 372 is already newer than 372 2023-06-27T12:45:11.834 INFO:tasks.rados.rados.0.smithi138.stdout:1713: oid 18 version 372 is already newer than 372 2023-06-27T12:45:11.834 INFO:tasks.rados.rados.0.smithi138.stdout:1713: oid 18 version 372 is already newer than 372 2023-06-27T12:45:11.834 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 18 v 372 (ObjNum 821 snap 199 seq_num 821) dirty exists 2023-06-27T12:45:11.834 INFO:tasks.rados.rados.0.smithi138.stdout:1713: left oid 18 (ObjNum 821 snap 199 seq_num 821) 2023-06-27T12:45:11.834 INFO:tasks.rados.rados.0.smithi138.stdout:1712: finishing write tid 5 to smithi138138488-28 2023-06-27T12:45:11.835 INFO:tasks.rados.rados.0.smithi138.stdout:1712: oid 28 updating version 0 to 317 2023-06-27T12:45:11.835 INFO:tasks.rados.rados.0.smithi138.stdout:1712: oid 28 version 317 is already newer than 317 2023-06-27T12:45:11.835 INFO:tasks.rados.rados.0.smithi138.stdout:1712: oid 28 version 317 is already newer than 317 2023-06-27T12:45:11.835 INFO:tasks.rados.rados.0.smithi138.stdout:1712: oid 28 version 317 is already newer than 317 2023-06-27T12:45:11.835 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 28 v 317 (ObjNum 820 snap 199 seq_num 820) dirty exists 2023-06-27T12:45:11.835 INFO:tasks.rados.rados.0.smithi138.stdout:1712: left oid 28 (ObjNum 820 snap 199 seq_num 820) 2023-06-27T12:45:11.835 INFO:tasks.rados.rados.0.smithi138.stdout:1715: expect (ObjNum 784 snap 192 seq_num 784) 2023-06-27T12:45:11.872 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:11.874 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:11.875 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:45:12.001 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:45:12.013 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":627,"stamp":"2023-06-27T12:45:11.618819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":123806731,"num_objects":443,"num_object_clones":396,"num_object_copies":1329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2575,"num_read_kb":1930039,"num_write":2334,"num_write_kb":683257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4345,"ondisk_log_size":4345,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1309440,"kb_used_data":1165280,"kb_used_omap":0,"kb_used_meta":144128,"kb_avail":376177920,"statfs":{"total":386547056640,"available":385206190080,"internally_reserved":0,"allocated":1193246720,"data_stored":1393510770,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":147587072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3445138,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":85,"num_read_kb":73053,"num_write":70,"num_write_kb":22909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023799"},"pg_stats":[{"pgid":"3.f","version":"412'75","reported_seq":994,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.627262+0000","last_change":"2023-06-27T12:45:08.697786+0000","last_active":"2023-06-27T12:45:08.627262+0000","last_peered":"2023-06-27T12:45:08.627262+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:45:08.627262+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680898,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":6253,"num_write":47,"num_write_kb":13642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"410'310","reported_seq":1306,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.696308+0000","last_change":"2023-06-27T12:45:08.827943+0000","last_active":"2023-06-27T12:45:08.696308+0000","last_peered":"2023-06-27T12:45:08.696308+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:45:08.696308+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9328380,"num_objects":37,"num_object_clones":33,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":176,"num_read_kb":217439,"num_write":194,"num_write_kb":58918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"411'359","reported_seq":1308,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.830009+0000","last_change":"2023-06-27T12:45:08.942761+0000","last_active":"2023-06-27T12:45:08.830009+0000","last_peered":"2023-06-27T12:45:08.830009+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:45:08.830009+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15715304,"num_objects":40,"num_object_clones":35,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":311,"num_read_kb":216081,"num_write":225,"num_write_kb":65874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"402'248","reported_seq":1226,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.615344+0000","last_change":"2023-06-27T12:45:08.629221+0000","last_active":"2023-06-27T12:45:08.615344+0000","last_peered":"2023-06-27T12:45:08.615344+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:45:08.615344+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874591,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":232,"num_read_kb":172087,"num_write":156,"num_write_kb":44779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"413'115","reported_seq":728,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.830604+0000","last_change":"2023-06-27T12:45:08.625370+0000","last_active":"2023-06-27T12:45:08.830604+0000","last_peered":"2023-06-27T12:45:08.830604+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:45:08.830604+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6655869,"num_objects":10,"num_object_clones":9,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":91084,"num_write":63,"num_write_kb":19283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"409'464","reported_seq":1293,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.620439+0000","last_change":"2023-06-27T12:45:08.693876+0000","last_active":"2023-06-27T12:45:08.620439+0000","last_peered":"2023-06-27T12:45:08.620439+0000","last_clean":"2023-06-27T12:45:08.620439+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:45:08.620439+0000","last_undegraded":"2023-06-27T12:45:08.620439+0000","last_fullsized":"2023-06-27T12:45:08.620439+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11479808,"num_objects":50,"num_object_clones":44,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":349,"num_read_kb":223353,"num_write":270,"num_write_kb":77359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"413'312","reported_seq":1130,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.990030+0000","last_change":"2023-06-27T12:45:08.977957+0000","last_active":"2023-06-27T12:45:08.990030+0000","last_peered":"2023-06-27T12:45:08.990030+0000","last_clean":"2023-06-27T12:45:08.990030+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:45:08.990030+0000","last_undegraded":"2023-06-27T12:45:08.990030+0000","last_fullsized":"2023-06-27T12:45:08.990030+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10790589,"num_objects":35,"num_object_clones":32,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":99991,"num_write":177,"num_write_kb":52146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"409'95","reported_seq":1027,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.660091+0000","last_change":"2023-06-27T12:45:08.756069+0000","last_active":"2023-06-27T12:45:08.660091+0000","last_peered":"2023-06-27T12:45:08.660091+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:45:08.660091+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395179,"num_objects":11,"num_object_clones":10,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":47130,"num_write":55,"num_write_kb":16677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":698,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.695263+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:45:08.695263+0000","last_peered":"2023-06-27T12:45:08.695263+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:45:08.695263+0000","last_undegraded":"2023-06-27T12:45:08.695263+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":923,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.665595+0000","last_change":"2023-06-27T12:45:08.704109+0000","last_active":"2023-06-27T12:45:08.665595+0000","last_peered":"2023-06-27T12:45:08.665595+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:45:08.665595+0000","last_undegraded":"2023-06-27T12:45:08.665595+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":673,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.621255+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:45:08.621255+0000","last_peered":"2023-06-27T12:45:08.621255+0000","last_clean":"2023-06-27T12:45:08.621255+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:45:08.621255+0000","last_undegraded":"2023-06-27T12:45:08.621255+0000","last_fullsized":"2023-06-27T12:45:08.621255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"412'239","reported_seq":1186,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.699102+0000","last_change":"2023-06-27T12:45:08.882790+0000","last_active":"2023-06-27T12:45:08.699102+0000","last_peered":"2023-06-27T12:45:08.699102+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:45:08.699102+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403995,"num_objects":26,"num_object_clones":21,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":101,"num_read_kb":90892,"num_write":145,"num_write_kb":40262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":666,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.672719+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:45:08.672719+0000","last_peered":"2023-06-27T12:45:08.672719+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:45:08.672719+0000","last_undegraded":"2023-06-27T12:45:08.672719+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"411'525","reported_seq":895,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.693823+0000","last_change":"2023-06-27T12:45:08.872720+0000","last_active":"2023-06-27T12:45:08.693823+0000","last_peered":"2023-06-27T12:45:08.693823+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:45:08.693823+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168532,"num_objects":33,"num_object_clones":31,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":99,"num_read_kb":75874,"num_write":172,"num_write_kb":53028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":33}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":663,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.662167+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:45:08.662167+0000","last_peered":"2023-06-27T12:45:08.662167+0000","last_clean":"2023-06-27T12:45:08.662167+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:45:08.662167+0000","last_undegraded":"2023-06-27T12:45:08.662167+0000","last_fullsized":"2023-06-27T12:45:08.662167+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":425,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.622415+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:45:08.622415+0000","last_peered":"2023-06-27T12:45:08.622415+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:45:08.622415+0000","last_undegraded":"2023-06-27T12:45:08.622415+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"412'498","reported_seq":1432,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.663330+0000","last_change":"2023-06-27T12:45:08.837221+0000","last_active":"2023-06-27T12:45:08.663330+0000","last_peered":"2023-06-27T12:45:08.663330+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:45:08.663330+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13446260,"num_objects":59,"num_object_clones":53,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":59,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":357,"num_read_kb":215334,"num_write":328,"num_write_kb":96236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":59}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":673,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.622910+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:45:08.622910+0000","last_peered":"2023-06-27T12:45:08.622910+0000","last_clean":"2023-06-27T12:45:08.622910+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:45:08.622910+0000","last_undegraded":"2023-06-27T12:45:08.622910+0000","last_fullsized":"2023-06-27T12:45:08.622910+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"393'286","reported_seq":808,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.663038+0000","last_change":"2023-06-27T12:45:08.699349+0000","last_active":"2023-06-27T12:45:08.663038+0000","last_peered":"2023-06-27T12:45:08.663038+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:45:08.663038+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5597127,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":177,"num_read_kb":120046,"num_write":122,"num_write_kb":34526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":441,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.688269+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:45:08.688269+0000","last_peered":"2023-06-27T12:45:08.688269+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:45:08.688269+0000","last_undegraded":"2023-06-27T12:45:08.688269+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"412'289","reported_seq":1250,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.692910+0000","last_change":"2023-06-27T12:45:08.835198+0000","last_active":"2023-06-27T12:45:08.692910+0000","last_peered":"2023-06-27T12:45:08.692910+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:45:08.692910+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9983853,"num_objects":35,"num_object_clones":31,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":142,"num_read_kb":155684,"num_write":178,"num_write_kb":51540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":35}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"406'174","reported_seq":731,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.657449+0000","last_change":"2023-06-27T12:45:08.665670+0000","last_active":"2023-06-27T12:45:08.657449+0000","last_peered":"2023-06-27T12:45:08.657449+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:45:08.657449+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3277530,"num_objects":13,"num_object_clones":13,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":112,"num_read_kb":62858,"num_write":67,"num_write_kb":19070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":13}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":698,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.695570+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:45:08.695570+0000","last_peered":"2023-06-27T12:45:08.695570+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:45:08.695570+0000","last_undegraded":"2023-06-27T12:45:08.695570+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":663,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.663199+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:45:08.663199+0000","last_peered":"2023-06-27T12:45:08.663199+0000","last_clean":"2023-06-27T12:45:08.663199+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:45:08.663199+0000","last_undegraded":"2023-06-27T12:45:08.663199+0000","last_fullsized":"2023-06-27T12:45:08.663199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"412'259","reported_seq":1205,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.693880+0000","last_change":"2023-06-27T12:45:08.808703+0000","last_active":"2023-06-27T12:45:08.693880+0000","last_peered":"2023-06-27T12:45:08.693880+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:45:08.693880+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7008816,"num_objects":28,"num_object_clones":25,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":128,"num_read_kb":135933,"num_write":135,"num_write_kb":39917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":123806731,"num_objects":443,"num_object_clones":396,"num_object_copies":1329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2575,"num_read_kb":1930039,"num_write":2334,"num_write_kb":683257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4248,"ondisk_log_size":4248,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673395,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":354844,"kb_used_data":319828,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94016996,"statfs":{"total":96636764160,"available":96273403904,"internally_reserved":0,"allocated":327503872,"data_stored":384747835,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706100,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":371828,"kb_used_data":332716,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":94000012,"statfs":{"total":96636764160,"available":96256012288,"internally_reserved":0,"allocated":340701184,"data_stored":397053720,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673395,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":401488,"kb_used_data":362376,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93970352,"statfs":{"total":96636764160,"available":96225640448,"internally_reserved":0,"allocated":371073024,"data_stored":434669663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:45:12.015 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:45:12.140 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:12.133+0000 7f1aaf614700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:12.143 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:12.139+0000 7f1aaf614700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:12.336 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:45:12.338 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:45:12.350 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2023-06-27T12:45:12.008832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":123806731,"num_objects":443,"num_object_clones":396,"num_object_copies":1329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2575,"num_read_kb":1930039,"num_write":2334,"num_write_kb":683257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4345,"ondisk_log_size":4345,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1309440,"kb_used_data":1165280,"kb_used_omap":0,"kb_used_meta":144128,"kb_avail":376177920,"statfs":{"total":386547056640,"available":385206190080,"internally_reserved":0,"allocated":1193246720,"data_stored":1393510770,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":147587072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3445138,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":85,"num_read_kb":73053,"num_write":70,"num_write_kb":22909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001045"},"pg_stats":[{"pgid":"3.f","version":"412'75","reported_seq":994,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.627262+0000","last_change":"2023-06-27T12:45:08.697786+0000","last_active":"2023-06-27T12:45:08.627262+0000","last_peered":"2023-06-27T12:45:08.627262+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:45:08.627262+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680898,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":6253,"num_write":47,"num_write_kb":13642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"410'310","reported_seq":1306,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.696308+0000","last_change":"2023-06-27T12:45:08.827943+0000","last_active":"2023-06-27T12:45:08.696308+0000","last_peered":"2023-06-27T12:45:08.696308+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:45:08.696308+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9328380,"num_objects":37,"num_object_clones":33,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":176,"num_read_kb":217439,"num_write":194,"num_write_kb":58918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"411'359","reported_seq":1308,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.830009+0000","last_change":"2023-06-27T12:45:08.942761+0000","last_active":"2023-06-27T12:45:08.830009+0000","last_peered":"2023-06-27T12:45:08.830009+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:45:08.830009+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15715304,"num_objects":40,"num_object_clones":35,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":311,"num_read_kb":216081,"num_write":225,"num_write_kb":65874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"402'248","reported_seq":1226,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.615344+0000","last_change":"2023-06-27T12:45:08.629221+0000","last_active":"2023-06-27T12:45:08.615344+0000","last_peered":"2023-06-27T12:45:08.615344+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:45:08.615344+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874591,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":232,"num_read_kb":172087,"num_write":156,"num_write_kb":44779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"413'115","reported_seq":728,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.830604+0000","last_change":"2023-06-27T12:45:08.625370+0000","last_active":"2023-06-27T12:45:08.830604+0000","last_peered":"2023-06-27T12:45:08.830604+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:45:08.830604+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6655869,"num_objects":10,"num_object_clones":9,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":91084,"num_write":63,"num_write_kb":19283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"409'464","reported_seq":1293,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.620439+0000","last_change":"2023-06-27T12:45:08.693876+0000","last_active":"2023-06-27T12:45:08.620439+0000","last_peered":"2023-06-27T12:45:08.620439+0000","last_clean":"2023-06-27T12:45:08.620439+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:45:08.620439+0000","last_undegraded":"2023-06-27T12:45:08.620439+0000","last_fullsized":"2023-06-27T12:45:08.620439+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11479808,"num_objects":50,"num_object_clones":44,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":349,"num_read_kb":223353,"num_write":270,"num_write_kb":77359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"413'312","reported_seq":1130,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.990030+0000","last_change":"2023-06-27T12:45:08.977957+0000","last_active":"2023-06-27T12:45:08.990030+0000","last_peered":"2023-06-27T12:45:08.990030+0000","last_clean":"2023-06-27T12:45:08.990030+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:45:08.990030+0000","last_undegraded":"2023-06-27T12:45:08.990030+0000","last_fullsized":"2023-06-27T12:45:08.990030+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10790589,"num_objects":35,"num_object_clones":32,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":99991,"num_write":177,"num_write_kb":52146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"409'95","reported_seq":1027,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.660091+0000","last_change":"2023-06-27T12:45:08.756069+0000","last_active":"2023-06-27T12:45:08.660091+0000","last_peered":"2023-06-27T12:45:08.660091+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:45:08.660091+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395179,"num_objects":11,"num_object_clones":10,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":47130,"num_write":55,"num_write_kb":16677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":698,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.695263+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:45:08.695263+0000","last_peered":"2023-06-27T12:45:08.695263+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:45:08.695263+0000","last_undegraded":"2023-06-27T12:45:08.695263+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":923,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.665595+0000","last_change":"2023-06-27T12:45:08.704109+0000","last_active":"2023-06-27T12:45:08.665595+0000","last_peered":"2023-06-27T12:45:08.665595+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:45:08.665595+0000","last_undegraded":"2023-06-27T12:45:08.665595+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":673,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.621255+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:45:08.621255+0000","last_peered":"2023-06-27T12:45:08.621255+0000","last_clean":"2023-06-27T12:45:08.621255+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:45:08.621255+0000","last_undegraded":"2023-06-27T12:45:08.621255+0000","last_fullsized":"2023-06-27T12:45:08.621255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"412'239","reported_seq":1186,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.699102+0000","last_change":"2023-06-27T12:45:08.882790+0000","last_active":"2023-06-27T12:45:08.699102+0000","last_peered":"2023-06-27T12:45:08.699102+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:45:08.699102+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403995,"num_objects":26,"num_object_clones":21,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":101,"num_read_kb":90892,"num_write":145,"num_write_kb":40262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":666,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.672719+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:45:08.672719+0000","last_peered":"2023-06-27T12:45:08.672719+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:45:08.672719+0000","last_undegraded":"2023-06-27T12:45:08.672719+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"411'525","reported_seq":895,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.693823+0000","last_change":"2023-06-27T12:45:08.872720+0000","last_active":"2023-06-27T12:45:08.693823+0000","last_peered":"2023-06-27T12:45:08.693823+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:45:08.693823+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168532,"num_objects":33,"num_object_clones":31,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":99,"num_read_kb":75874,"num_write":172,"num_write_kb":53028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":33}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":663,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.662167+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:45:08.662167+0000","last_peered":"2023-06-27T12:45:08.662167+0000","last_clean":"2023-06-27T12:45:08.662167+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:45:08.662167+0000","last_undegraded":"2023-06-27T12:45:08.662167+0000","last_fullsized":"2023-06-27T12:45:08.662167+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":425,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.622415+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:45:08.622415+0000","last_peered":"2023-06-27T12:45:08.622415+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:45:08.622415+0000","last_undegraded":"2023-06-27T12:45:08.622415+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"412'498","reported_seq":1432,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.663330+0000","last_change":"2023-06-27T12:45:08.837221+0000","last_active":"2023-06-27T12:45:08.663330+0000","last_peered":"2023-06-27T12:45:08.663330+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:45:08.663330+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13446260,"num_objects":59,"num_object_clones":53,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":59,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":357,"num_read_kb":215334,"num_write":328,"num_write_kb":96236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":59}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":673,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.622910+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:45:08.622910+0000","last_peered":"2023-06-27T12:45:08.622910+0000","last_clean":"2023-06-27T12:45:08.622910+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:45:08.622910+0000","last_undegraded":"2023-06-27T12:45:08.622910+0000","last_fullsized":"2023-06-27T12:45:08.622910+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"393'286","reported_seq":808,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.663038+0000","last_change":"2023-06-27T12:45:08.699349+0000","last_active":"2023-06-27T12:45:08.663038+0000","last_peered":"2023-06-27T12:45:08.663038+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:45:08.663038+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5597127,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":177,"num_read_kb":120046,"num_write":122,"num_write_kb":34526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":441,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.688269+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:45:08.688269+0000","last_peered":"2023-06-27T12:45:08.688269+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:45:08.688269+0000","last_undegraded":"2023-06-27T12:45:08.688269+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"412'289","reported_seq":1250,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.692910+0000","last_change":"2023-06-27T12:45:08.835198+0000","last_active":"2023-06-27T12:45:08.692910+0000","last_peered":"2023-06-27T12:45:08.692910+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:45:08.692910+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9983853,"num_objects":35,"num_object_clones":31,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":142,"num_read_kb":155684,"num_write":178,"num_write_kb":51540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":35}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"406'174","reported_seq":731,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.657449+0000","last_change":"2023-06-27T12:45:08.665670+0000","last_active":"2023-06-27T12:45:08.657449+0000","last_peered":"2023-06-27T12:45:08.657449+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:45:08.657449+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3277530,"num_objects":13,"num_object_clones":13,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":112,"num_read_kb":62858,"num_write":67,"num_write_kb":19070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":13}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":698,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.695570+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:45:08.695570+0000","last_peered":"2023-06-27T12:45:08.695570+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:45:08.695570+0000","last_undegraded":"2023-06-27T12:45:08.695570+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":663,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.663199+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:45:08.663199+0000","last_peered":"2023-06-27T12:45:08.663199+0000","last_clean":"2023-06-27T12:45:08.663199+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:45:08.663199+0000","last_undegraded":"2023-06-27T12:45:08.663199+0000","last_fullsized":"2023-06-27T12:45:08.663199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"412'259","reported_seq":1205,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.693880+0000","last_change":"2023-06-27T12:45:08.808703+0000","last_active":"2023-06-27T12:45:08.693880+0000","last_peered":"2023-06-27T12:45:08.693880+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:45:08.693880+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7008816,"num_objects":28,"num_object_clones":25,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":128,"num_read_kb":135933,"num_write":135,"num_write_kb":39917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":123806731,"num_objects":443,"num_object_clones":396,"num_object_copies":1329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2575,"num_read_kb":1930039,"num_write":2334,"num_write_kb":683257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4248,"ondisk_log_size":4248,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673395,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":354844,"kb_used_data":319828,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94016996,"statfs":{"total":96636764160,"available":96273403904,"internally_reserved":0,"allocated":327503872,"data_stored":384747835,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706100,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":371828,"kb_used_data":332716,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":94000012,"statfs":{"total":96636764160,"available":96256012288,"internally_reserved":0,"allocated":340701184,"data_stored":397053720,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673395,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":401488,"kb_used_data":362376,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93970352,"statfs":{"total":96636764160,"available":96225640448,"internally_reserved":0,"allocated":371073024,"data_stored":434669663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:45:12.351 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-06-27T12:45:12.352 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:45:12.352 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:45:12.478 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:12.470+0000 7fecc8a97700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:12.481 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:12.477+0000 7fecc8a97700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:12.630 INFO:tasks.rados.rados.0.smithi138.stdout:1710: done (5 left) 2023-06-27T12:45:12.630 INFO:tasks.rados.rados.0.smithi138.stdout:1711: done (4 left) 2023-06-27T12:45:12.630 INFO:tasks.rados.rados.0.smithi138.stdout:1712: done (3 left) 2023-06-27T12:45:12.630 INFO:tasks.rados.rados.0.smithi138.stdout:1713: done (2 left) 2023-06-27T12:45:12.631 INFO:tasks.rados.rados.0.smithi138.stdout:1715: done (1 left) 2023-06-27T12:45:12.631 INFO:tasks.rados.rados.0.smithi138.stdout:1716: done (0 left) 2023-06-27T12:45:12.631 INFO:tasks.rados.rados.0.smithi138.stdout:1717: write oid 26 current snap is 200 2023-06-27T12:45:12.631 INFO:tasks.rados.rados.0.smithi138.stdout:1717: seq_num 822 ranges {615354=420802,1651375=751957,3190053=492771} 2023-06-27T12:45:12.638 INFO:tasks.rados.rados.0.smithi138.stdout:1717: writing smithi138138488-26 from 615354 to 1036156 tid 1 2023-06-27T12:45:12.650 INFO:tasks.rados.rados.0.smithi138.stdout:1717: writing smithi138138488-26 from 1651375 to 2403332 tid 2 2023-06-27T12:45:12.659 INFO:tasks.rados.rados.0.smithi138.stdout:1717: writing smithi138138488-26 from 3190053 to 3682824 tid 3 2023-06-27T12:45:12.660 INFO:tasks.rados.rados.0.smithi138.stdout:1718: delete oid 47 current snap is 200 2023-06-27T12:45:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:45:12.696 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:45:12.711 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":629,"stamp":"2023-06-27T12:45:12.622348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":123806731,"num_objects":443,"num_object_clones":396,"num_object_copies":1329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2575,"num_read_kb":1930039,"num_write":2334,"num_write_kb":683257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4345,"ondisk_log_size":4345,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1309440,"kb_used_data":1165280,"kb_used_omap":0,"kb_used_meta":144128,"kb_avail":376177920,"statfs":{"total":386547056640,"available":385206190080,"internally_reserved":0,"allocated":1193246720,"data_stored":1393510770,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":147587072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3445138,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":85,"num_read_kb":73053,"num_write":70,"num_write_kb":22909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013991"},"pg_stats":[{"pgid":"3.f","version":"412'75","reported_seq":994,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.627262+0000","last_change":"2023-06-27T12:45:08.697786+0000","last_active":"2023-06-27T12:45:08.627262+0000","last_peered":"2023-06-27T12:45:08.627262+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:45:08.627262+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680898,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":6253,"num_write":47,"num_write_kb":13642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"410'310","reported_seq":1306,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.696308+0000","last_change":"2023-06-27T12:45:08.827943+0000","last_active":"2023-06-27T12:45:08.696308+0000","last_peered":"2023-06-27T12:45:08.696308+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:45:08.696308+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9328380,"num_objects":37,"num_object_clones":33,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":176,"num_read_kb":217439,"num_write":194,"num_write_kb":58918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"411'359","reported_seq":1308,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.830009+0000","last_change":"2023-06-27T12:45:08.942761+0000","last_active":"2023-06-27T12:45:08.830009+0000","last_peered":"2023-06-27T12:45:08.830009+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:45:08.830009+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15715304,"num_objects":40,"num_object_clones":35,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":311,"num_read_kb":216081,"num_write":225,"num_write_kb":65874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"402'248","reported_seq":1226,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.615344+0000","last_change":"2023-06-27T12:45:08.629221+0000","last_active":"2023-06-27T12:45:08.615344+0000","last_peered":"2023-06-27T12:45:08.615344+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:45:08.615344+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874591,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":232,"num_read_kb":172087,"num_write":156,"num_write_kb":44779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"413'115","reported_seq":728,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.830604+0000","last_change":"2023-06-27T12:45:08.625370+0000","last_active":"2023-06-27T12:45:08.830604+0000","last_peered":"2023-06-27T12:45:08.830604+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:45:08.830604+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6655869,"num_objects":10,"num_object_clones":9,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":91084,"num_write":63,"num_write_kb":19283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"409'464","reported_seq":1293,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.620439+0000","last_change":"2023-06-27T12:45:08.693876+0000","last_active":"2023-06-27T12:45:08.620439+0000","last_peered":"2023-06-27T12:45:08.620439+0000","last_clean":"2023-06-27T12:45:08.620439+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:45:08.620439+0000","last_undegraded":"2023-06-27T12:45:08.620439+0000","last_fullsized":"2023-06-27T12:45:08.620439+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11479808,"num_objects":50,"num_object_clones":44,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":349,"num_read_kb":223353,"num_write":270,"num_write_kb":77359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"413'312","reported_seq":1130,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.990030+0000","last_change":"2023-06-27T12:45:08.977957+0000","last_active":"2023-06-27T12:45:08.990030+0000","last_peered":"2023-06-27T12:45:08.990030+0000","last_clean":"2023-06-27T12:45:08.990030+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:45:08.990030+0000","last_undegraded":"2023-06-27T12:45:08.990030+0000","last_fullsized":"2023-06-27T12:45:08.990030+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10790589,"num_objects":35,"num_object_clones":32,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":99991,"num_write":177,"num_write_kb":52146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"409'95","reported_seq":1027,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.660091+0000","last_change":"2023-06-27T12:45:08.756069+0000","last_active":"2023-06-27T12:45:08.660091+0000","last_peered":"2023-06-27T12:45:08.660091+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:45:08.660091+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395179,"num_objects":11,"num_object_clones":10,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":47130,"num_write":55,"num_write_kb":16677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":698,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.695263+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:45:08.695263+0000","last_peered":"2023-06-27T12:45:08.695263+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:45:08.695263+0000","last_undegraded":"2023-06-27T12:45:08.695263+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":923,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.665595+0000","last_change":"2023-06-27T12:45:08.704109+0000","last_active":"2023-06-27T12:45:08.665595+0000","last_peered":"2023-06-27T12:45:08.665595+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:45:08.665595+0000","last_undegraded":"2023-06-27T12:45:08.665595+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":673,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.621255+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:45:08.621255+0000","last_peered":"2023-06-27T12:45:08.621255+0000","last_clean":"2023-06-27T12:45:08.621255+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:45:08.621255+0000","last_undegraded":"2023-06-27T12:45:08.621255+0000","last_fullsized":"2023-06-27T12:45:08.621255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"412'239","reported_seq":1186,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.699102+0000","last_change":"2023-06-27T12:45:08.882790+0000","last_active":"2023-06-27T12:45:08.699102+0000","last_peered":"2023-06-27T12:45:08.699102+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:45:08.699102+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403995,"num_objects":26,"num_object_clones":21,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":101,"num_read_kb":90892,"num_write":145,"num_write_kb":40262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":666,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.672719+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:45:08.672719+0000","last_peered":"2023-06-27T12:45:08.672719+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:45:08.672719+0000","last_undegraded":"2023-06-27T12:45:08.672719+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"411'525","reported_seq":895,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.693823+0000","last_change":"2023-06-27T12:45:08.872720+0000","last_active":"2023-06-27T12:45:08.693823+0000","last_peered":"2023-06-27T12:45:08.693823+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:45:08.693823+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6168532,"num_objects":33,"num_object_clones":31,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":99,"num_read_kb":75874,"num_write":172,"num_write_kb":53028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":33}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":663,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.662167+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:45:08.662167+0000","last_peered":"2023-06-27T12:45:08.662167+0000","last_clean":"2023-06-27T12:45:08.662167+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:45:08.662167+0000","last_undegraded":"2023-06-27T12:45:08.662167+0000","last_fullsized":"2023-06-27T12:45:08.662167+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":425,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.622415+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:45:08.622415+0000","last_peered":"2023-06-27T12:45:08.622415+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:45:08.622415+0000","last_undegraded":"2023-06-27T12:45:08.622415+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"412'498","reported_seq":1432,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.663330+0000","last_change":"2023-06-27T12:45:08.837221+0000","last_active":"2023-06-27T12:45:08.663330+0000","last_peered":"2023-06-27T12:45:08.663330+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:45:08.663330+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13446260,"num_objects":59,"num_object_clones":53,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":59,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":357,"num_read_kb":215334,"num_write":328,"num_write_kb":96236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":59}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":673,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.622910+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:45:08.622910+0000","last_peered":"2023-06-27T12:45:08.622910+0000","last_clean":"2023-06-27T12:45:08.622910+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:45:08.622910+0000","last_undegraded":"2023-06-27T12:45:08.622910+0000","last_fullsized":"2023-06-27T12:45:08.622910+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"393'286","reported_seq":808,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.663038+0000","last_change":"2023-06-27T12:45:08.699349+0000","last_active":"2023-06-27T12:45:08.663038+0000","last_peered":"2023-06-27T12:45:08.663038+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:45:08.663038+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5597127,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":177,"num_read_kb":120046,"num_write":122,"num_write_kb":34526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":441,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.688269+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:45:08.688269+0000","last_peered":"2023-06-27T12:45:08.688269+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:45:08.688269+0000","last_undegraded":"2023-06-27T12:45:08.688269+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"412'289","reported_seq":1250,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.692910+0000","last_change":"2023-06-27T12:45:08.835198+0000","last_active":"2023-06-27T12:45:08.692910+0000","last_peered":"2023-06-27T12:45:08.692910+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:45:08.692910+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9983853,"num_objects":35,"num_object_clones":31,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":142,"num_read_kb":155684,"num_write":178,"num_write_kb":51540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":35}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"406'174","reported_seq":731,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.657449+0000","last_change":"2023-06-27T12:45:08.665670+0000","last_active":"2023-06-27T12:45:08.657449+0000","last_peered":"2023-06-27T12:45:08.657449+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:45:08.657449+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3277530,"num_objects":13,"num_object_clones":13,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":112,"num_read_kb":62858,"num_write":67,"num_write_kb":19070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":13}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":698,"reported_epoch":413,"state":"active+undersized","last_fresh":"2023-06-27T12:45:08.695570+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:45:08.695570+0000","last_peered":"2023-06-27T12:45:08.695570+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:45:08.695570+0000","last_undegraded":"2023-06-27T12:45:08.695570+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":663,"reported_epoch":413,"state":"active+clean","last_fresh":"2023-06-27T12:45:08.663199+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:45:08.663199+0000","last_peered":"2023-06-27T12:45:08.663199+0000","last_clean":"2023-06-27T12:45:08.663199+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:45:08.663199+0000","last_undegraded":"2023-06-27T12:45:08.663199+0000","last_fullsized":"2023-06-27T12:45:08.663199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"412'259","reported_seq":1205,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:08.693880+0000","last_change":"2023-06-27T12:45:08.808703+0000","last_active":"2023-06-27T12:45:08.693880+0000","last_peered":"2023-06-27T12:45:08.693880+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:45:08.693880+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7008816,"num_objects":28,"num_object_clones":25,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":128,"num_read_kb":135933,"num_write":135,"num_write_kb":39917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":123806731,"num_objects":443,"num_object_clones":396,"num_object_copies":1329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2575,"num_read_kb":1930039,"num_write":2334,"num_write_kb":683257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4248,"ondisk_log_size":4248,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673395,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":354844,"kb_used_data":319828,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94016996,"statfs":{"total":96636764160,"available":96273403904,"internally_reserved":0,"allocated":327503872,"data_stored":384747835,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706100,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":371828,"kb_used_data":332716,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":94000012,"statfs":{"total":96636764160,"available":96256012288,"internally_reserved":0,"allocated":340701184,"data_stored":397053720,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673395,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":401488,"kb_used_data":362376,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93970352,"statfs":{"total":96636764160,"available":96225640448,"internally_reserved":0,"allocated":371073024,"data_stored":434669663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:45:12.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.e 3.2 3.1 3.7 2023-06-27T12:45:12.721 INFO:tasks.rados.rados.0.smithi138.stdout:1718: done (1 left) 2023-06-27T12:45:12.722 INFO:tasks.rados.rados.0.smithi138.stdout:1719: snap_create 2023-06-27T12:45:12.722 INFO:tasks.rados.rados.0.smithi138.stdout:1717: finishing write tid 1 to smithi138138488-26 2023-06-27T12:45:12.767 INFO:tasks.rados.rados.0.smithi138.stdout:1717: finishing write tid 2 to smithi138138488-26 2023-06-27T12:45:12.768 INFO:tasks.rados.rados.0.smithi138.stdout:1717: finishing write tid 3 to smithi138138488-26 2023-06-27T12:45:12.768 INFO:tasks.rados.rados.0.smithi138.stdout:1717: finishing write tid 4 to smithi138138488-26 2023-06-27T12:45:12.769 INFO:tasks.rados.rados.0.smithi138.stdout:1717: finishing write tid 5 to smithi138138488-26 2023-06-27T12:45:12.769 INFO:tasks.rados.rados.0.smithi138.stdout:1717: oid 26 updating version 0 to 304 2023-06-27T12:45:12.770 INFO:tasks.rados.rados.0.smithi138.stdout:1717: oid 26 version 304 is already newer than 304 2023-06-27T12:45:12.770 INFO:tasks.rados.rados.0.smithi138.stdout:1717: oid 26 version 304 is already newer than 304 2023-06-27T12:45:12.770 INFO:tasks.rados.rados.0.smithi138.stdout:1717: oid 26 version 304 is already newer than 304 2023-06-27T12:45:12.770 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 26 v 304 (ObjNum 822 snap 200 seq_num 822) dirty exists 2023-06-27T12:45:12.770 INFO:tasks.rados.rados.0.smithi138.stdout:1717: left oid 26 (ObjNum 822 snap 200 seq_num 822) 2023-06-27T12:45:12.849 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:12.838+0000 7f37cecb5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:12.853 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:12.849+0000 7f37cecb5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:13.049 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.1,3.2,3.7] on osd.1 to force-recovery; instructing pg(s) [3.e] on osd.2 to force-recovery; 2023-06-27T12:45:13.050 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:45:13.631 INFO:tasks.rados.rados.0.smithi138.stdout:1717: done (1 left) 2023-06-27T12:45:13.631 INFO:tasks.rados.rados.0.smithi138.stdout:1719: done (0 left) 2023-06-27T12:45:13.632 INFO:tasks.rados.rados.0.smithi138.stdout:1720: write oid 21 current snap is 201 2023-06-27T12:45:13.632 INFO:tasks.rados.rados.0.smithi138.stdout:1720: seq_num 824 ranges {648968=571300,1651986=639273,2728546=631668} 2023-06-27T12:45:13.639 INFO:tasks.rados.rados.0.smithi138.stdout:1720: writing smithi138138488-21 from 648968 to 1220268 tid 1 2023-06-27T12:45:13.645 INFO:tasks.rados.rados.0.smithi138.stdout:1720: writing smithi138138488-21 from 1651986 to 2291259 tid 2 2023-06-27T12:45:13.652 INFO:tasks.rados.rados.0.smithi138.stdout:1720: writing smithi138138488-21 from 2728546 to 3360214 tid 3 2023-06-27T12:45:13.652 INFO:tasks.rados.rados.0.smithi138.stdout:1721: write (excl) oid 3 current snap is 201 2023-06-27T12:45:13.652 INFO:tasks.rados.rados.0.smithi138.stdout:1721: seq_num 825 ranges {665775=646549,2110468=499284,3155969=442940} 2023-06-27T12:45:13.659 INFO:tasks.rados.rados.0.smithi138.stdout:1721: writing smithi138138488-3 from 665775 to 1312324 tid 1 2023-06-27T12:45:13.666 INFO:tasks.rados.rados.0.smithi138.stdout:1721: writing smithi138138488-3 from 2110468 to 2609752 tid 2 2023-06-27T12:45:13.672 INFO:tasks.rados.rados.0.smithi138.stdout:1721: writing smithi138138488-3 from 3155969 to 3598909 tid 3 2023-06-27T12:45:13.672 INFO:tasks.rados.rados.0.smithi138.stdout:1722: write (excl) oid 37 current snap is 201 2023-06-27T12:45:13.672 INFO:tasks.rados.rados.0.smithi138.stdout:1722: seq_num 826 ranges {682582=721798,2168950=642942,3583392=299190} 2023-06-27T12:45:13.681 INFO:tasks.rados.rados.0.smithi138.stdout:1722: writing smithi138138488-37 from 682582 to 1404380 tid 1 2023-06-27T12:45:13.690 INFO:tasks.rados.rados.0.smithi138.stdout:1722: writing smithi138138488-37 from 2168950 to 2811892 tid 2 2023-06-27T12:45:13.697 INFO:tasks.rados.rados.0.smithi138.stdout:1722: writing smithi138138488-37 from 3583392 to 3882582 tid 3 2023-06-27T12:45:13.697 INFO:tasks.rados.rados.0.smithi138.stdout:1723: snap_create 2023-06-27T12:45:13.697 INFO:tasks.rados.rados.0.smithi138.stdout:1720: finishing write tid 1 to smithi138138488-21 2023-06-27T12:45:13.719 INFO:tasks.rados.rados.0.smithi138.stdout:1721: finishing write tid 1 to smithi138138488-3 2023-06-27T12:45:13.733 INFO:tasks.rados.rados.0.smithi138.stdout:1720: finishing write tid 2 to smithi138138488-21 2023-06-27T12:45:13.737 INFO:tasks.rados.rados.0.smithi138.stdout:1720: finishing write tid 3 to smithi138138488-21 2023-06-27T12:45:13.751 INFO:tasks.rados.rados.0.smithi138.stdout:1720: finishing write tid 4 to smithi138138488-21 2023-06-27T12:45:13.752 INFO:tasks.rados.rados.0.smithi138.stdout:1720: finishing write tid 5 to smithi138138488-21 2023-06-27T12:45:13.752 INFO:tasks.rados.rados.0.smithi138.stdout:1720: oid 21 updating version 0 to 517 2023-06-27T12:45:13.752 INFO:tasks.rados.rados.0.smithi138.stdout:1720: oid 21 version 517 is already newer than 517 2023-06-27T12:45:13.752 INFO:tasks.rados.rados.0.smithi138.stdout:1720: oid 21 version 517 is already newer than 517 2023-06-27T12:45:13.752 INFO:tasks.rados.rados.0.smithi138.stdout:1720: oid 21 version 517 is already newer than 517 2023-06-27T12:45:13.752 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 21 v 517 (ObjNum 824 snap 201 seq_num 824) dirty exists 2023-06-27T12:45:13.753 INFO:tasks.rados.rados.0.smithi138.stdout:1720: left oid 21 (ObjNum 824 snap 201 seq_num 824) 2023-06-27T12:45:13.753 INFO:tasks.rados.rados.0.smithi138.stdout:1721: finishing write tid 2 to smithi138138488-3 2023-06-27T12:45:13.754 INFO:tasks.rados.rados.0.smithi138.stdout:1722: finishing write tid 1 to smithi138138488-37 2023-06-27T12:45:13.780 INFO:tasks.rados.rados.0.smithi138.stdout:1722: finishing write tid 2 to smithi138138488-37 2023-06-27T12:45:13.798 INFO:tasks.rados.rados.0.smithi138.stdout:1721: finishing write tid 3 to smithi138138488-3 2023-06-27T12:45:13.798 INFO:tasks.rados.rados.0.smithi138.stdout:1721: finishing write tid 4 to smithi138138488-3 2023-06-27T12:45:13.800 INFO:tasks.rados.rados.0.smithi138.stdout:1721: finishing write tid 5 to smithi138138488-3 2023-06-27T12:45:13.800 INFO:tasks.rados.rados.0.smithi138.stdout:1721: oid 3 updating version 0 to 309 2023-06-27T12:45:13.800 INFO:tasks.rados.rados.0.smithi138.stdout:1721: oid 3 version 309 is already newer than 309 2023-06-27T12:45:13.800 INFO:tasks.rados.rados.0.smithi138.stdout:1721: oid 3 version 309 is already newer than 305 2023-06-27T12:45:13.800 INFO:tasks.rados.rados.0.smithi138.stdout:1721: oid 3 version 309 is already newer than 309 2023-06-27T12:45:13.801 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 309 (ObjNum 825 snap 201 seq_num 825) dirty exists 2023-06-27T12:45:13.801 INFO:tasks.rados.rados.0.smithi138.stdout:1721: left oid 3 (ObjNum 825 snap 201 seq_num 825) 2023-06-27T12:45:13.820 INFO:tasks.rados.rados.0.smithi138.stdout:1722: finishing write tid 3 to smithi138138488-37 2023-06-27T12:45:13.821 INFO:tasks.rados.rados.0.smithi138.stdout:1722: finishing write tid 4 to smithi138138488-37 2023-06-27T12:45:13.822 INFO:tasks.rados.rados.0.smithi138.stdout:1722: finishing write tid 5 to smithi138138488-37 2023-06-27T12:45:13.822 INFO:tasks.rados.rados.0.smithi138.stdout:1722: oid 37 updating version 0 to 381 2023-06-27T12:45:13.823 INFO:tasks.rados.rados.0.smithi138.stdout:1722: oid 37 version 381 is already newer than 377 2023-06-27T12:45:13.823 INFO:tasks.rados.rados.0.smithi138.stdout:1722: oid 37 version 381 is already newer than 381 2023-06-27T12:45:13.823 INFO:tasks.rados.rados.0.smithi138.stdout:1722: oid 37 version 381 is already newer than 381 2023-06-27T12:45:13.823 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 381 (ObjNum 826 snap 201 seq_num 826) dirty exists 2023-06-27T12:45:13.823 INFO:tasks.rados.rados.0.smithi138.stdout:1722: left oid 37 (ObjNum 826 snap 201 seq_num 826) 2023-06-27T12:45:13.877 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:13.997+0000 7f637194a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:14.005 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:14.001+0000 7f637194a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:14.051 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:14.054 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:14.055 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:14.640 INFO:tasks.rados.rados.0.smithi138.stdout:1720: done (3 left) 2023-06-27T12:45:14.640 INFO:tasks.rados.rados.0.smithi138.stdout:1721: done (2 left) 2023-06-27T12:45:14.640 INFO:tasks.rados.rados.0.smithi138.stdout:1722: done (1 left) 2023-06-27T12:45:14.640 INFO:tasks.rados.rados.0.smithi138.stdout:1723: done (0 left) 2023-06-27T12:45:14.641 INFO:tasks.rados.rados.0.smithi138.stdout:1724: snap_create 2023-06-27T12:45:15.474 INFO:tasks.rados.rados.0.smithi138.stdout:1724: done (0 left) 2023-06-27T12:45:15.474 INFO:tasks.rados.rados.0.smithi138.stdout:1725: write (excl) oid 19 current snap is 203 2023-06-27T12:45:15.474 INFO:tasks.rados.rados.0.smithi138.stdout:1725: seq_num 827 ranges {699389=797047,2111079=502953,3094462=581837} 2023-06-27T12:45:15.482 INFO:tasks.rados.rados.0.smithi138.stdout:1725: writing smithi138138488-19 from 699389 to 1496436 tid 1 2023-06-27T12:45:15.490 INFO:tasks.rados.rados.0.smithi138.stdout:1725: writing smithi138138488-19 from 2111079 to 2614032 tid 2 2023-06-27T12:45:15.495 INFO:tasks.rados.rados.0.smithi138.stdout:1725: writing smithi138138488-19 from 3094462 to 3676299 tid 3 2023-06-27T12:45:15.495 INFO:tasks.rados.rados.0.smithi138.stdout:1726: delete oid 5 current snap is 203 2023-06-27T12:45:15.559 INFO:tasks.rados.rados.0.smithi138.stdout:1725: finishing write tid 1 to smithi138138488-19 2023-06-27T12:45:15.580 INFO:tasks.rados.rados.0.smithi138.stdout:1726: done (1 left) 2023-06-27T12:45:15.581 INFO:tasks.rados.rados.0.smithi138.stdout:1727: snap_remove snap 200 2023-06-27T12:45:16.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~281s 2023-06-27T12:45:16.056 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:45:16.186 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:16.181+0000 7f302a35d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:16.189 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:16.186+0000 7f302a35d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:16.235 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:16.238 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:16.238 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:16.479 INFO:tasks.rados.rados.0.smithi138.stdout:1727: done (1 left) 2023-06-27T12:45:16.480 INFO:tasks.rados.rados.0.smithi138.stdout:1728: write (excl) oid 25 current snap is 203 2023-06-27T12:45:16.480 INFO:tasks.rados.rados.0.smithi138.stdout:1725: finishing write tid 2 to smithi138138488-19 2023-06-27T12:45:16.480 INFO:tasks.rados.rados.0.smithi138.stdout:1728: seq_num 829 ranges {733003=663898,1944396=506622,2865661=604381} 2023-06-27T12:45:16.487 INFO:tasks.rados.rados.0.smithi138.stdout:1728: writing smithi138138488-25 from 733003 to 1396901 tid 1 2023-06-27T12:45:16.493 INFO:tasks.rados.rados.0.smithi138.stdout:1728: writing smithi138138488-25 from 1944396 to 2451018 tid 2 2023-06-27T12:45:16.499 INFO:tasks.rados.rados.0.smithi138.stdout:1728: writing smithi138138488-25 from 2865661 to 3470042 tid 3 2023-06-27T12:45:16.499 INFO:tasks.rados.rados.0.smithi138.stdout:1729: read oid 37 snap -1 2023-06-27T12:45:16.500 INFO:tasks.rados.rados.0.smithi138.stdout:1729: expect (ObjNum 826 snap 201 seq_num 826) 2023-06-27T12:45:16.500 INFO:tasks.rados.rados.0.smithi138.stdout:1730: snap_remove snap 199 2023-06-27T12:45:17.480 INFO:tasks.rados.rados.0.smithi138.stdout:1730: done (3 left) 2023-06-27T12:45:17.480 INFO:tasks.rados.rados.0.smithi138.stdout:1731: delete oid 50 current snap is 203 2023-06-27T12:45:17.480 INFO:tasks.rados.rados.0.smithi138.stdout:1725: finishing write tid 3 to smithi138138488-19 2023-06-27T12:45:17.480 INFO:tasks.rados.rados.0.smithi138.stdout:1725: finishing write tid 4 to smithi138138488-19 2023-06-27T12:45:17.480 INFO:tasks.rados.rados.0.smithi138.stdout:1725: finishing write tid 5 to smithi138138488-19 2023-06-27T12:45:17.481 INFO:tasks.rados.rados.0.smithi138.stdout:1725: oid 19 updating version 0 to 522 2023-06-27T12:45:17.481 INFO:tasks.rados.rados.0.smithi138.stdout:1725: oid 19 version 522 is already newer than 522 2023-06-27T12:45:17.481 INFO:tasks.rados.rados.0.smithi138.stdout:1725: oid 19 version 522 is already newer than 522 2023-06-27T12:45:17.481 INFO:tasks.rados.rados.0.smithi138.stdout:1725: oid 19 version 522 is already newer than 518 2023-06-27T12:45:17.481 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 522 (ObjNum 827 snap 203 seq_num 827) dirty exists 2023-06-27T12:45:17.481 INFO:tasks.rados.rados.0.smithi138.stdout:1725: left oid 19 (ObjNum 827 snap 203 seq_num 827) 2023-06-27T12:45:17.481 INFO:tasks.rados.rados.0.smithi138.stdout:1728: finishing write tid 1 to smithi138138488-25 2023-06-27T12:45:17.481 INFO:tasks.rados.rados.0.smithi138.stdout:1728: finishing write tid 2 to smithi138138488-25 2023-06-27T12:45:17.482 INFO:tasks.rados.rados.0.smithi138.stdout:1729: expect (ObjNum 826 snap 201 seq_num 826) 2023-06-27T12:45:17.692 INFO:tasks.rados.rados.0.smithi138.stdout:1728: finishing write tid 3 to smithi138138488-25 2023-06-27T12:45:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:1728: finishing write tid 4 to smithi138138488-25 2023-06-27T12:45:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:1728: finishing write tid 5 to smithi138138488-25 2023-06-27T12:45:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:1728: oid 25 updating version 0 to 185 2023-06-27T12:45:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:1728: oid 25 version 185 is already newer than 181 2023-06-27T12:45:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:1728: oid 25 version 185 is already newer than 185 2023-06-27T12:45:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:1728: oid 25 version 185 is already newer than 185 2023-06-27T12:45:17.693 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 25 v 185 (ObjNum 829 snap 203 seq_num 829) dirty exists 2023-06-27T12:45:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:1728: left oid 25 (ObjNum 829 snap 203 seq_num 829) 2023-06-27T12:45:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:1725: done (3 left) 2023-06-27T12:45:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:1728: done (2 left) 2023-06-27T12:45:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:1729: done (1 left) 2023-06-27T12:45:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:1731: done (0 left) 2023-06-27T12:45:17.694 INFO:tasks.rados.rados.0.smithi138.stdout:1732: snap_remove snap 197 2023-06-27T12:45:18.061 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:45:18.062 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:45:18.062 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:45:18.189 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:18.181+0000 7fe499dbd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:18.192 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:18.188+0000 7fe499dbd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:18.240 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:18.366 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:18.361+0000 7f074c16b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:18.369 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:18.365+0000 7f074c16b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:18.396 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:45:18.398 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:45:18.410 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":637,"stamp":"2023-06-27T12:45:18.010862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":127033675,"num_objects":450,"num_object_clones":403,"num_object_copies":1350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2606,"num_read_kb":1948160,"num_write":2369,"num_write_kb":694947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4417,"ondisk_log_size":4417,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1327660,"kb_used_data":1183500,"kb_used_omap":0,"kb_used_meta":144128,"kb_avail":376159700,"statfs":{"total":386547056640,"available":385187532800,"internally_reserved":0,"allocated":1211904000,"data_stored":1417363459,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":147587072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5882615,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":27,"num_read_kb":18119,"num_write":25,"num_write_kb":8509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001356"},"pg_stats":[{"pgid":"3.f","version":"412'75","reported_seq":1002,"reported_epoch":416,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:11.638612+0000","last_change":"2023-06-27T12:45:11.642904+0000","last_active":"2023-06-27T12:45:11.638612+0000","last_peered":"2023-06-27T12:45:11.638612+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:45:11.638612+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680898,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":6253,"num_write":47,"num_write_kb":13642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"410'310","reported_seq":1318,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.639972+0000","last_change":"2023-06-27T12:45:12.740624+0000","last_active":"2023-06-27T12:45:13.639972+0000","last_peered":"2023-06-27T12:45:13.639972+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:45:13.639972+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9328380,"num_objects":37,"num_object_clones":33,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":177,"num_read_kb":217439,"num_write":194,"num_write_kb":58918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"418'381","reported_seq":1328,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.767547+0000","last_change":"2023-06-27T12:45:12.667032+0000","last_active":"2023-06-27T12:45:13.767547+0000","last_peered":"2023-06-27T12:45:13.767547+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:45:13.767547+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16959022,"num_objects":42,"num_object_clones":37,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":314,"num_read_kb":216083,"num_write":235,"num_write_kb":69262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"402'248","reported_seq":1238,"reported_epoch":416,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:11.685748+0000","last_change":"2023-06-27T12:45:11.652768+0000","last_active":"2023-06-27T12:45:11.685748+0000","last_peered":"2023-06-27T12:45:11.685748+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:45:11.685748+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874591,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":239,"num_read_kb":179662,"num_write":156,"num_write_kb":44779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"413'115","reported_seq":739,"reported_epoch":416,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:11.646602+0000","last_change":"2023-06-27T12:45:11.695428+0000","last_active":"2023-06-27T12:45:11.646602+0000","last_peered":"2023-06-27T12:45:11.646602+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:45:11.646602+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6655869,"num_objects":10,"num_object_clones":9,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":96067,"num_write":63,"num_write_kb":19283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"409'464","reported_seq":1302,"reported_epoch":416,"state":"active+clean","last_fresh":"2023-06-27T12:45:11.641246+0000","last_change":"2023-06-27T12:45:11.692550+0000","last_active":"2023-06-27T12:45:11.641246+0000","last_peered":"2023-06-27T12:45:11.641246+0000","last_clean":"2023-06-27T12:45:11.641246+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:45:11.641246+0000","last_undegraded":"2023-06-27T12:45:11.641246+0000","last_fullsized":"2023-06-27T12:45:11.641246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11479808,"num_objects":50,"num_object_clones":44,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":349,"num_read_kb":223353,"num_write":270,"num_write_kb":77359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"415'317","reported_seq":1144,"reported_epoch":416,"state":"active+clean","last_fresh":"2023-06-27T12:45:11.635703+0000","last_change":"2023-06-27T12:45:11.697582+0000","last_active":"2023-06-27T12:45:11.635703+0000","last_peered":"2023-06-27T12:45:11.635703+0000","last_clean":"2023-06-27T12:45:11.635703+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:45:11.635703+0000","last_undegraded":"2023-06-27T12:45:11.635703+0000","last_fullsized":"2023-06-27T12:45:11.635703+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11009080,"num_objects":36,"num_object_clones":33,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":99992,"num_write":182,"num_write_kb":53878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"409'95","reported_seq":1038,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.656278+0000","last_change":"2023-06-27T12:45:12.644242+0000","last_active":"2023-06-27T12:45:13.656278+0000","last_peered":"2023-06-27T12:45:13.656278+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:45:13.656278+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395179,"num_objects":11,"num_object_clones":10,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":47130,"num_write":55,"num_write_kb":16677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":706,"reported_epoch":418,"state":"active+undersized","last_fresh":"2023-06-27T12:45:13.638759+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:45:13.638759+0000","last_peered":"2023-06-27T12:45:13.638759+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:45:13.638759+0000","last_undegraded":"2023-06-27T12:45:13.638759+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":933,"reported_epoch":418,"state":"active+undersized","last_fresh":"2023-06-27T12:45:13.650599+0000","last_change":"2023-06-27T12:45:12.648277+0000","last_active":"2023-06-27T12:45:13.650599+0000","last_peered":"2023-06-27T12:45:13.650599+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:45:13.650599+0000","last_undegraded":"2023-06-27T12:45:13.650599+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":679,"reported_epoch":416,"state":"active+clean","last_fresh":"2023-06-27T12:45:11.640508+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:45:11.640508+0000","last_peered":"2023-06-27T12:45:11.640508+0000","last_clean":"2023-06-27T12:45:11.640508+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:45:11.640508+0000","last_undegraded":"2023-06-27T12:45:11.640508+0000","last_fullsized":"2023-06-27T12:45:11.640508+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"412'239","reported_seq":1197,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.642484+0000","last_change":"2023-06-27T12:45:12.719044+0000","last_active":"2023-06-27T12:45:13.642484+0000","last_peered":"2023-06-27T12:45:13.642484+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:45:13.642484+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403995,"num_objects":26,"num_object_clones":21,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":101,"num_read_kb":90892,"num_write":145,"num_write_kb":40262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":26}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":674,"reported_epoch":418,"state":"active+undersized","last_fresh":"2023-06-27T12:45:13.646489+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:45:13.646489+0000","last_peered":"2023-06-27T12:45:13.646489+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:45:13.646489+0000","last_undegraded":"2023-06-27T12:45:13.646489+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"415'531","reported_seq":907,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.638354+0000","last_change":"2023-06-27T12:45:12.742667+0000","last_active":"2023-06-27T12:45:13.638354+0000","last_peered":"2023-06-27T12:45:13.638354+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:45:13.638354+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722932,"num_objects":34,"num_object_clones":32,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":99,"num_read_kb":75874,"num_write":172,"num_write_kb":53028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":34}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":671,"reported_epoch":418,"state":"active+clean","last_fresh":"2023-06-27T12:45:13.646956+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:45:13.646956+0000","last_peered":"2023-06-27T12:45:13.646956+0000","last_clean":"2023-06-27T12:45:13.646956+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:45:13.646956+0000","last_undegraded":"2023-06-27T12:45:13.646956+0000","last_fullsized":"2023-06-27T12:45:13.646956+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":431,"reported_epoch":416,"state":"active+undersized","last_fresh":"2023-06-27T12:45:11.645126+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:45:11.645126+0000","last_peered":"2023-06-27T12:45:11.645126+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:45:11.645126+0000","last_undegraded":"2023-06-27T12:45:11.645126+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"418'517","reported_seq":1453,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.690119+0000","last_change":"2023-06-27T12:45:12.850234+0000","last_active":"2023-06-27T12:45:13.690119+0000","last_peered":"2023-06-27T12:45:13.690119+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:45:13.690119+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17866666,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":359,"num_read_kb":215336,"num_write":338,"num_write_kb":99625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":679,"reported_epoch":416,"state":"active+clean","last_fresh":"2023-06-27T12:45:11.643568+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:45:11.643568+0000","last_peered":"2023-06-27T12:45:11.643568+0000","last_clean":"2023-06-27T12:45:11.643568+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:45:11.643568+0000","last_undegraded":"2023-06-27T12:45:11.643568+0000","last_fullsized":"2023-06-27T12:45:11.643568+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"393'286","reported_seq":819,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.654554+0000","last_change":"2023-06-27T12:45:12.649298+0000","last_active":"2023-06-27T12:45:13.654554+0000","last_peered":"2023-06-27T12:45:13.654554+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:45:13.654554+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5597127,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":177,"num_read_kb":120046,"num_write":122,"num_write_kb":34526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":449,"reported_epoch":418,"state":"active+undersized","last_fresh":"2023-06-27T12:45:13.638147+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:45:13.638147+0000","last_peered":"2023-06-27T12:45:13.638147+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:45:13.638147+0000","last_undegraded":"2023-06-27T12:45:13.638147+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"418'309","reported_seq":1271,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.722551+0000","last_change":"2023-06-27T12:45:12.711590+0000","last_active":"2023-06-27T12:45:13.722551+0000","last_peered":"2023-06-27T12:45:13.722551+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:45:13.722551+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10773782,"num_objects":37,"num_object_clones":33,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":145,"num_read_kb":155686,"num_write":188,"num_write_kb":54721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"406'174","reported_seq":744,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.650406+0000","last_change":"2023-06-27T12:45:12.670502+0000","last_active":"2023-06-27T12:45:13.650406+0000","last_peered":"2023-06-27T12:45:13.650406+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:45:13.650406+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3277530,"num_objects":13,"num_object_clones":13,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":119,"num_read_kb":68414,"num_write":67,"num_write_kb":19070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":13}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":706,"reported_epoch":418,"state":"active+undersized","last_fresh":"2023-06-27T12:45:13.647704+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:45:13.647704+0000","last_peered":"2023-06-27T12:45:13.647704+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:45:13.647704+0000","last_undegraded":"2023-06-27T12:45:13.647704+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":671,"reported_epoch":418,"state":"active+clean","last_fresh":"2023-06-27T12:45:13.651423+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:45:13.651423+0000","last_peered":"2023-06-27T12:45:13.651423+0000","last_clean":"2023-06-27T12:45:13.651423+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:45:13.651423+0000","last_undegraded":"2023-06-27T12:45:13.651423+0000","last_fullsized":"2023-06-27T12:45:13.651423+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"412'259","reported_seq":1216,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:13.635771+0000","last_change":"2023-06-27T12:45:12.701649+0000","last_active":"2023-06-27T12:45:13.635771+0000","last_peered":"2023-06-27T12:45:13.635771+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:45:13.635771+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7008816,"num_objects":28,"num_object_clones":25,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":128,"num_read_kb":135933,"num_write":135,"num_write_kb":39917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":127033675,"num_objects":450,"num_object_clones":403,"num_object_copies":1350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":17,"num_read":2606,"num_read_kb":1948160,"num_write":2369,"num_write_kb":694947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4320,"ondisk_log_size":4320,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673400,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":359824,"kb_used_data":324808,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94012016,"statfs":{"total":96636764160,"available":96268304384,"internally_reserved":0,"allocated":332603392,"data_stored":392501624,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706105,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":380040,"kb_used_data":340928,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93991800,"statfs":{"total":96636764160,"available":96247603200,"internally_reserved":0,"allocated":349110272,"data_stored":407826696,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673400,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":406516,"kb_used_data":367404,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93965324,"statfs":{"total":96636764160,"available":96220491776,"internally_reserved":0,"allocated":376221696,"data_stored":439995587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:45:18.412 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.d 3.c 3.3 3.0 3.2 3.1 3.6 2023-06-27T12:45:18.415 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:18.418 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:18.418 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:18.548 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:18.533+0000 7f73101ea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:18.551 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:18.547+0000 7f73101ea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:18.650 INFO:tasks.rados.rados.0.smithi138.stdout:1732: done (0 left) 2023-06-27T12:45:18.650 INFO:tasks.rados.rados.0.smithi138.stdout:1733: read oid 48 snap -1 2023-06-27T12:45:18.650 INFO:tasks.rados.rados.0.smithi138.stdout:1733: expect deleted 2023-06-27T12:45:18.651 INFO:tasks.rados.rados.0.smithi138.stdout:1734: read oid 28 snap -1 2023-06-27T12:45:18.651 INFO:tasks.rados.rados.0.smithi138.stdout:1734: expect (ObjNum 820 snap 199 seq_num 820) 2023-06-27T12:45:18.651 INFO:tasks.rados.rados.0.smithi138.stdout:1735: write (excl) oid 6 current snap is 203 2023-06-27T12:45:18.651 INFO:tasks.rados.rados.0.smithi138.stdout:1735: seq_num 831 ranges {766617=414396,1945007=510291,3204154=626925} 2023-06-27T12:45:18.656 INFO:tasks.rados.rados.0.smithi138.stdout:1735: writing smithi138138488-6 from 766617 to 1181013 tid 1 2023-06-27T12:45:18.664 INFO:tasks.rados.rados.0.smithi138.stdout:1735: writing smithi138138488-6 from 1945007 to 2455298 tid 2 2023-06-27T12:45:18.671 INFO:tasks.rados.rados.0.smithi138.stdout:1735: writing smithi138138488-6 from 3204154 to 3831079 tid 3 2023-06-27T12:45:18.672 INFO:tasks.rados.rados.0.smithi138.stdout:1736: snap_remove snap 192 2023-06-27T12:45:18.849 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.c,3.f] on osd.0 to force-backfill; instructing pg(s) [3.1,3.2,3.d] on osd.1 to force-backfill; instructing pg(s) [3.0,3.3,3.6] on osd.2 to force-backfill; 2023-06-27T12:45:18.850 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:45:19.659 INFO:tasks.rados.rados.0.smithi138.stdout:1736: done (3 left) 2023-06-27T12:45:19.660 INFO:tasks.rados.rados.0.smithi138.stdout:1737: write oid 15 current snap is 203 2023-06-27T12:45:19.660 INFO:tasks.rados.rados.0.smithi138.stdout:1737: seq_num 832 ranges {783424=489645,2003489=653949,3231577=554550} 2023-06-27T12:45:19.666 INFO:tasks.rados.rados.0.smithi138.stdout:1737: writing smithi138138488-15 from 783424 to 1273069 tid 1 2023-06-27T12:45:19.674 INFO:tasks.rados.rados.0.smithi138.stdout:1737: writing smithi138138488-15 from 2003489 to 2657438 tid 2 2023-06-27T12:45:19.681 INFO:tasks.rados.rados.0.smithi138.stdout:1737: writing smithi138138488-15 from 3231577 to 3786127 tid 3 2023-06-27T12:45:19.681 INFO:tasks.rados.rados.0.smithi138.stdout:1738: write oid 9 current snap is 203 2023-06-27T12:45:19.682 INFO:tasks.rados.rados.0.smithi138.stdout:1738: seq_num 833 ranges {400231=564894,1661971=338260} 2023-06-27T12:45:19.686 INFO:tasks.rados.rados.0.smithi138.stdout:1738: writing smithi138138488-9 from 400231 to 965125 tid 1 2023-06-27T12:45:19.693 INFO:tasks.rados.rados.0.smithi138.stdout:1738: writing smithi138138488-9 from 1661971 to 2000231 tid 2 2023-06-27T12:45:19.693 INFO:tasks.rados.rados.0.smithi138.stdout:1739: write (excl) oid 20 current snap is 203 2023-06-27T12:45:19.693 INFO:tasks.rados.rados.0.smithi138.stdout:1734: expect (ObjNum 820 snap 199 seq_num 820) 2023-06-27T12:45:19.950 INFO:tasks.rados.rados.0.smithi138.stdout:1735: finishing write tid 1 to smithi138138488-6 2023-06-27T12:45:19.950 INFO:tasks.rados.rados.0.smithi138.stdout:1735: finishing write tid 2 to smithi138138488-6 2023-06-27T12:45:19.951 INFO:tasks.rados.rados.0.smithi138.stdout:1735: finishing write tid 3 to smithi138138488-6 2023-06-27T12:45:19.951 INFO:tasks.rados.rados.0.smithi138.stdout:1735: finishing write tid 4 to smithi138138488-6 2023-06-27T12:45:19.951 INFO:tasks.rados.rados.0.smithi138.stdout:1735: finishing write tid 5 to smithi138138488-6 2023-06-27T12:45:19.951 INFO:tasks.rados.rados.0.smithi138.stdout:1735: oid 6 updating version 0 to 250 2023-06-27T12:45:19.951 INFO:tasks.rados.rados.0.smithi138.stdout:1735: oid 6 version 250 is already newer than 246 2023-06-27T12:45:19.951 INFO:tasks.rados.rados.0.smithi138.stdout:1735: oid 6 version 250 is already newer than 250 2023-06-27T12:45:19.952 INFO:tasks.rados.rados.0.smithi138.stdout:1735: oid 6 version 250 is already newer than 250 2023-06-27T12:45:19.952 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 250 (ObjNum 831 snap 203 seq_num 831) dirty exists 2023-06-27T12:45:19.952 INFO:tasks.rados.rados.0.smithi138.stdout:1735: left oid 6 (ObjNum 831 snap 203 seq_num 831) 2023-06-27T12:45:19.952 INFO:tasks.rados.rados.0.smithi138.stdout:1737: finishing write tid 1 to smithi138138488-15 2023-06-27T12:45:19.952 INFO:tasks.rados.rados.0.smithi138.stdout:1739: seq_num 834 ranges {417038=640143,1720453=296585} 2023-06-27T12:45:19.955 INFO:tasks.rados.rados.0.smithi138.stdout:1739: writing smithi138138488-20 from 417038 to 1057181 tid 1 2023-06-27T12:45:19.961 INFO:tasks.rados.rados.0.smithi138.stdout:1739: writing smithi138138488-20 from 1720453 to 2017038 tid 2 2023-06-27T12:45:19.961 INFO:tasks.rados.rados.0.smithi138.stdout:1733: done (5 left) 2023-06-27T12:45:19.961 INFO:tasks.rados.rados.0.smithi138.stdout:1734: done (4 left) 2023-06-27T12:45:19.961 INFO:tasks.rados.rados.0.smithi138.stdout:1735: done (3 left) 2023-06-27T12:45:19.961 INFO:tasks.rados.rados.0.smithi138.stdout:1740: write (excl) oid 6 current snap is 203 2023-06-27T12:45:19.961 INFO:tasks.rados.rados.0.smithi138.stdout:1737: finishing write tid 2 to smithi138138488-15 2023-06-27T12:45:19.961 INFO:tasks.rados.rados.0.smithi138.stdout:1737: finishing write tid 3 to smithi138138488-15 2023-06-27T12:45:19.962 INFO:tasks.rados.rados.0.smithi138.stdout:1740: seq_num 835 ranges {433845=715392,1662582=371263} 2023-06-27T12:45:19.967 INFO:tasks.rados.rados.0.smithi138.stdout:1740: writing smithi138138488-6 from 433845 to 1149237 tid 1 2023-06-27T12:45:19.972 INFO:tasks.rados.rados.0.smithi138.stdout:1740: writing smithi138138488-6 from 1662582 to 2033845 tid 2 2023-06-27T12:45:19.972 INFO:tasks.rados.rados.0.smithi138.stdout:1741: write (excl) oid 38 current snap is 203 2023-06-27T12:45:19.972 INFO:tasks.rados.rados.0.smithi138.stdout:1741: seq_num 836 ranges {450652=790641,1721064=329588} 2023-06-27T12:45:19.978 INFO:tasks.rados.rados.0.smithi138.stdout:1741: writing smithi138138488-38 from 450652 to 1241293 tid 1 2023-06-27T12:45:19.983 INFO:tasks.rados.rados.0.smithi138.stdout:1741: writing smithi138138488-38 from 1721064 to 2050652 tid 2 2023-06-27T12:45:19.983 INFO:tasks.rados.rados.0.smithi138.stdout:1742: snap_create 2023-06-27T12:45:19.983 INFO:tasks.rados.rados.0.smithi138.stdout:1737: finishing write tid 4 to smithi138138488-15 2023-06-27T12:45:19.984 INFO:tasks.rados.rados.0.smithi138.stdout:1737: finishing write tid 5 to smithi138138488-15 2023-06-27T12:45:19.984 INFO:tasks.rados.rados.0.smithi138.stdout:1737: oid 15 updating version 0 to 125 2023-06-27T12:45:19.984 INFO:tasks.rados.rados.0.smithi138.stdout:1737: oid 15 version 125 is already newer than 124 2023-06-27T12:45:19.985 INFO:tasks.rados.rados.0.smithi138.stdout:1737: oid 15 updating version 125 to 126 2023-06-27T12:45:19.985 INFO:tasks.rados.rados.0.smithi138.stdout:1737: oid 15 version 126 is already newer than 126 2023-06-27T12:45:19.985 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 15 v 126 (ObjNum 832 snap 203 seq_num 832) dirty exists 2023-06-27T12:45:19.985 INFO:tasks.rados.rados.0.smithi138.stdout:1737: left oid 15 (ObjNum 832 snap 203 seq_num 832) 2023-06-27T12:45:19.985 INFO:tasks.rados.rados.0.smithi138.stdout:1738: finishing write tid 1 to smithi138138488-9 2023-06-27T12:45:19.985 INFO:tasks.rados.rados.0.smithi138.stdout:1738: finishing write tid 2 to smithi138138488-9 2023-06-27T12:45:19.985 INFO:tasks.rados.rados.0.smithi138.stdout:1738: finishing write tid 3 to smithi138138488-9 2023-06-27T12:45:19.985 INFO:tasks.rados.rados.0.smithi138.stdout:1738: finishing write tid 4 to smithi138138488-9 2023-06-27T12:45:19.986 INFO:tasks.rados.rados.0.smithi138.stdout:1738: oid 9 updating version 0 to 540 2023-06-27T12:45:19.986 INFO:tasks.rados.rados.0.smithi138.stdout:1738: oid 9 version 540 is already newer than 540 2023-06-27T12:45:19.986 INFO:tasks.rados.rados.0.smithi138.stdout:1738: oid 9 version 540 is already newer than 540 2023-06-27T12:45:19.986 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 540 (ObjNum 833 snap 203 seq_num 833) dirty exists 2023-06-27T12:45:19.986 INFO:tasks.rados.rados.0.smithi138.stdout:1738: left oid 9 (ObjNum 833 snap 203 seq_num 833) 2023-06-27T12:45:20.006 INFO:tasks.rados.rados.0.smithi138.stdout:1741: finishing write tid 1 to smithi138138488-38 2023-06-27T12:45:20.034 INFO:tasks.rados.rados.0.smithi138.stdout:1740: finishing write tid 1 to smithi138138488-6 2023-06-27T12:45:20.059 INFO:tasks.rados.rados.0.smithi138.stdout:1741: finishing write tid 2 to smithi138138488-38 2023-06-27T12:45:20.059 INFO:tasks.rados.rados.0.smithi138.stdout:1741: finishing write tid 3 to smithi138138488-38 2023-06-27T12:45:20.060 INFO:tasks.rados.rados.0.smithi138.stdout:1741: finishing write tid 4 to smithi138138488-38 2023-06-27T12:45:20.060 INFO:tasks.rados.rados.0.smithi138.stdout:1741: oid 38 updating version 0 to 83 2023-06-27T12:45:20.061 INFO:tasks.rados.rados.0.smithi138.stdout:1741: oid 38 version 83 is already newer than 80 2023-06-27T12:45:20.061 INFO:tasks.rados.rados.0.smithi138.stdout:1741: oid 38 version 83 is already newer than 83 2023-06-27T12:45:20.061 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 38 v 83 (ObjNum 836 snap 203 seq_num 836) dirty exists 2023-06-27T12:45:20.061 INFO:tasks.rados.rados.0.smithi138.stdout:1741: left oid 38 (ObjNum 836 snap 203 seq_num 836) 2023-06-27T12:45:20.069 INFO:tasks.rados.rados.0.smithi138.stdout:1739: finishing write tid 1 to smithi138138488-20 2023-06-27T12:45:20.081 INFO:tasks.rados.rados.0.smithi138.stdout:1739: finishing write tid 2 to smithi138138488-20 2023-06-27T12:45:20.085 INFO:tasks.rados.rados.0.smithi138.stdout:1739: finishing write tid 3 to smithi138138488-20 2023-06-27T12:45:20.086 INFO:tasks.rados.rados.0.smithi138.stdout:1740: finishing write tid 2 to smithi138138488-6 2023-06-27T12:45:20.087 INFO:tasks.rados.rados.0.smithi138.stdout:1739: finishing write tid 4 to smithi138138488-20 2023-06-27T12:45:20.087 INFO:tasks.rados.rados.0.smithi138.stdout:1739: oid 20 updating version 0 to 322 2023-06-27T12:45:20.088 INFO:tasks.rados.rados.0.smithi138.stdout:1739: oid 20 version 322 is already newer than 319 2023-06-27T12:45:20.088 INFO:tasks.rados.rados.0.smithi138.stdout:1739: oid 20 version 322 is already newer than 322 2023-06-27T12:45:20.088 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 322 (ObjNum 834 snap 203 seq_num 834) dirty exists 2023-06-27T12:45:20.088 INFO:tasks.rados.rados.0.smithi138.stdout:1739: left oid 20 (ObjNum 834 snap 203 seq_num 834) 2023-06-27T12:45:20.090 INFO:tasks.rados.rados.0.smithi138.stdout:1740: finishing write tid 3 to smithi138138488-6 2023-06-27T12:45:20.096 INFO:tasks.rados.rados.0.smithi138.stdout:1740: finishing write tid 4 to smithi138138488-6 2023-06-27T12:45:20.096 INFO:tasks.rados.rados.0.smithi138.stdout:1740: oid 6 updating version 0 to 253 2023-06-27T12:45:20.096 INFO:tasks.rados.rados.0.smithi138.stdout:1740: oid 6 version 253 is already newer than 251 2023-06-27T12:45:20.097 INFO:tasks.rados.rados.0.smithi138.stdout:1740: oid 6 version 253 is already newer than 253 2023-06-27T12:45:20.097 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 6 v 253 (ObjNum 835 snap 203 seq_num 835) dirty exists 2023-06-27T12:45:20.097 INFO:tasks.rados.rados.0.smithi138.stdout:1740: left oid 6 (ObjNum 835 snap 203 seq_num 835) 2023-06-27T12:45:20.420 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:45:20.546 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:20.541+0000 7fa861c25700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:20.549 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:20.545+0000 7fa861c25700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:20.598 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:20.599 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:20.662 INFO:tasks.rados.rados.0.smithi138.stdout:1737: done (5 left) 2023-06-27T12:45:20.662 INFO:tasks.rados.rados.0.smithi138.stdout:1738: done (4 left) 2023-06-27T12:45:20.662 INFO:tasks.rados.rados.0.smithi138.stdout:1739: done (3 left) 2023-06-27T12:45:20.663 INFO:tasks.rados.rados.0.smithi138.stdout:1740: done (2 left) 2023-06-27T12:45:20.663 INFO:tasks.rados.rados.0.smithi138.stdout:1741: done (1 left) 2023-06-27T12:45:20.663 INFO:tasks.rados.rados.0.smithi138.stdout:1742: done (0 left) 2023-06-27T12:45:20.663 INFO:tasks.rados.rados.0.smithi138.stdout:1743: write (excl) oid 3 current snap is 204 2023-06-27T12:45:20.663 INFO:tasks.rados.rados.0.smithi138.stdout:1743: seq_num 837 ranges {467459=582243,1495899=404945} 2023-06-27T12:45:20.667 INFO:tasks.rados.rados.0.smithi138.stdout:1743: writing smithi138138488-3 from 467459 to 1049702 tid 1 2023-06-27T12:45:20.672 INFO:tasks.rados.rados.0.smithi138.stdout:1743: writing smithi138138488-3 from 1495899 to 1900844 tid 2 2023-06-27T12:45:20.672 INFO:tasks.rados.rados.0.smithi138.stdout:1744: delete oid 15 current snap is 204 2023-06-27T12:45:20.730 INFO:tasks.rados.rados.0.smithi138.stdout:1743: finishing write tid 1 to smithi138138488-3 2023-06-27T12:45:20.730 INFO:tasks.rados.rados.0.smithi138.stdout:1744: done (1 left) 2023-06-27T12:45:20.731 INFO:tasks.rados.rados.0.smithi138.stdout:1745: write oid 30 current snap is 204 2023-06-27T12:45:20.731 INFO:tasks.rados.rados.0.smithi138.stdout:1745: seq_num 839 ranges {501073=732741,1896510=204563} 2023-06-27T12:45:20.737 INFO:tasks.rados.rados.0.smithi138.stdout:1745: writing smithi138138488-30 from 501073 to 1233814 tid 1 2023-06-27T12:45:20.741 INFO:tasks.rados.rados.0.smithi138.stdout:1745: writing smithi138138488-30 from 1896510 to 2101073 tid 2 2023-06-27T12:45:20.742 INFO:tasks.rados.rados.0.smithi138.stdout:1746: delete oid 10 current snap is 204 2023-06-27T12:45:20.766 INFO:tasks.rados.rados.0.smithi138.stdout:1743: finishing write tid 2 to smithi138138488-3 2023-06-27T12:45:20.767 INFO:tasks.rados.rados.0.smithi138.stdout:1743: finishing write tid 3 to smithi138138488-3 2023-06-27T12:45:20.770 INFO:tasks.rados.rados.0.smithi138.stdout:1743: finishing write tid 4 to smithi138138488-3 2023-06-27T12:45:20.770 INFO:tasks.rados.rados.0.smithi138.stdout:1743: oid 3 updating version 0 to 319 2023-06-27T12:45:20.771 INFO:tasks.rados.rados.0.smithi138.stdout:1743: oid 3 version 319 is already newer than 319 2023-06-27T12:45:20.771 INFO:tasks.rados.rados.0.smithi138.stdout:1743: oid 3 version 319 is already newer than 316 2023-06-27T12:45:20.771 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 3 v 319 (ObjNum 837 snap 204 seq_num 837) dirty exists 2023-06-27T12:45:20.771 INFO:tasks.rados.rados.0.smithi138.stdout:1743: left oid 3 (ObjNum 837 snap 204 seq_num 837) 2023-06-27T12:45:20.807 INFO:tasks.rados.rados.0.smithi138.stdout:1743: done (2 left) 2023-06-27T12:45:20.807 INFO:tasks.rados.rados.0.smithi138.stdout:1746: done (1 left) 2023-06-27T12:45:20.808 INFO:tasks.rados.rados.0.smithi138.stdout:1747: snap_remove snap 196 2023-06-27T12:45:21.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~286s 2023-06-27T12:45:21.666 INFO:tasks.rados.rados.0.smithi138.stdout:1747: done (1 left) 2023-06-27T12:45:21.666 INFO:tasks.rados.rados.0.smithi138.stdout:1748: read oid 33 snap -1 2023-06-27T12:45:21.666 INFO:tasks.rados.rados.0.smithi138.stdout:1748: expect (ObjNum 766 snap 189 seq_num 766) 2023-06-27T12:45:21.667 INFO:tasks.rados.rados.0.smithi138.stdout:1749: write (excl) oid 9 current snap is 204 2023-06-27T12:45:21.667 INFO:tasks.rados.rados.0.smithi138.stdout:1745: finishing write tid 1 to smithi138138488-30 2023-06-27T12:45:21.667 INFO:tasks.rados.rados.0.smithi138.stdout:1745: finishing write tid 2 to smithi138138488-30 2023-06-27T12:45:21.667 INFO:tasks.rados.rados.0.smithi138.stdout:1745: finishing write tid 3 to smithi138138488-30 2023-06-27T12:45:21.667 INFO:tasks.rados.rados.0.smithi138.stdout:1745: finishing write tid 4 to smithi138138488-30 2023-06-27T12:45:21.667 INFO:tasks.rados.rados.0.smithi138.stdout:1745: oid 30 updating version 0 to 312 2023-06-27T12:45:21.667 INFO:tasks.rados.rados.0.smithi138.stdout:1745: oid 30 version 312 is already newer than 312 2023-06-27T12:45:21.667 INFO:tasks.rados.rados.0.smithi138.stdout:1745: oid 30 version 312 is already newer than 312 2023-06-27T12:45:21.668 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 312 (ObjNum 839 snap 204 seq_num 839) dirty exists 2023-06-27T12:45:21.668 INFO:tasks.rados.rados.0.smithi138.stdout:1745: left oid 30 (ObjNum 839 snap 204 seq_num 839) 2023-06-27T12:45:21.668 INFO:tasks.rados.rados.0.smithi138.stdout:1749: seq_num 841 ranges {534687=483239,1613474=412283} 2023-06-27T12:45:21.671 INFO:tasks.rados.rados.0.smithi138.stdout:1749: writing smithi138138488-9 from 534687 to 1017926 tid 1 2023-06-27T12:45:21.678 INFO:tasks.rados.rados.0.smithi138.stdout:1749: writing smithi138138488-9 from 1613474 to 2025757 tid 2 2023-06-27T12:45:21.678 INFO:tasks.rados.rados.0.smithi138.stdout:1745: done (2 left) 2023-06-27T12:45:21.678 INFO:tasks.rados.rados.0.smithi138.stdout:1750: delete oid 12 current snap is 204 2023-06-27T12:45:21.786 INFO:tasks.rados.rados.0.smithi138.stdout:1748: expect (ObjNum 766 snap 189 seq_num 766) 2023-06-27T12:45:22.010 INFO:tasks.rados.rados.0.smithi138.stdout:1749: finishing write tid 1 to smithi138138488-9 2023-06-27T12:45:22.010 INFO:tasks.rados.rados.0.smithi138.stdout:1749: finishing write tid 2 to smithi138138488-9 2023-06-27T12:45:22.011 INFO:tasks.rados.rados.0.smithi138.stdout:1749: finishing write tid 3 to smithi138138488-9 2023-06-27T12:45:22.011 INFO:tasks.rados.rados.0.smithi138.stdout:1749: finishing write tid 4 to smithi138138488-9 2023-06-27T12:45:22.011 INFO:tasks.rados.rados.0.smithi138.stdout:1749: oid 9 updating version 0 to 550 2023-06-27T12:45:22.011 INFO:tasks.rados.rados.0.smithi138.stdout:1749: oid 9 version 550 is already newer than 550 2023-06-27T12:45:22.011 INFO:tasks.rados.rados.0.smithi138.stdout:1749: oid 9 version 550 is already newer than 547 2023-06-27T12:45:22.011 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 9 v 550 (ObjNum 841 snap 204 seq_num 841) dirty exists 2023-06-27T12:45:22.011 INFO:tasks.rados.rados.0.smithi138.stdout:1749: left oid 9 (ObjNum 841 snap 204 seq_num 841) 2023-06-27T12:45:22.011 INFO:tasks.rados.rados.0.smithi138.stdout:1748: done (2 left) 2023-06-27T12:45:22.012 INFO:tasks.rados.rados.0.smithi138.stdout:1749: done (1 left) 2023-06-27T12:45:22.012 INFO:tasks.rados.rados.0.smithi138.stdout:1750: done (0 left) 2023-06-27T12:45:22.012 INFO:tasks.rados.rados.0.smithi138.stdout:1751: read oid 35 snap 198 2023-06-27T12:45:22.012 INFO:tasks.rados.rados.0.smithi138.stdout:1751: expect (ObjNum 788 snap 192 seq_num 788) 2023-06-27T12:45:22.012 INFO:tasks.rados.rados.0.smithi138.stdout:1752: write (excl) oid 43 current snap is 204 2023-06-27T12:45:22.012 INFO:tasks.rados.rados.0.smithi138.stdout:1752: seq_num 843 ranges {568301=633737,1614085=415952} 2023-06-27T12:45:22.016 INFO:tasks.rados.rados.0.smithi138.stdout:1752: writing smithi138138488-43 from 568301 to 1202038 tid 1 2023-06-27T12:45:22.021 INFO:tasks.rados.rados.0.smithi138.stdout:1752: writing smithi138138488-43 from 1614085 to 2030037 tid 2 2023-06-27T12:45:22.022 INFO:tasks.rados.rados.0.smithi138.stdout:1753: read oid 20 snap -1 2023-06-27T12:45:22.022 INFO:tasks.rados.rados.0.smithi138.stdout:1753: expect (ObjNum 834 snap 203 seq_num 834) 2023-06-27T12:45:22.022 INFO:tasks.rados.rados.0.smithi138.stdout:1754: write oid 24 current snap is 204 2023-06-27T12:45:22.022 INFO:tasks.rados.rados.0.smithi138.stdout:1754: seq_num 844 ranges {585108=425339,1788920=396188} 2023-06-27T12:45:22.027 INFO:tasks.rados.rados.0.smithi138.stdout:1754: writing smithi138138488-24 from 585108 to 1010447 tid 1 2023-06-27T12:45:22.034 INFO:tasks.rados.rados.0.smithi138.stdout:1754: writing smithi138138488-24 from 1788920 to 2185108 tid 2 2023-06-27T12:45:22.034 INFO:tasks.rados.rados.0.smithi138.stdout:1755: write oid 47 current snap is 204 2023-06-27T12:45:22.034 INFO:tasks.rados.rados.0.smithi138.stdout:1755: seq_num 845 ranges {601915=500588,1847402=354513} 2023-06-27T12:45:22.040 INFO:tasks.rados.rados.0.smithi138.stdout:1755: writing smithi138138488-47 from 601915 to 1102503 tid 1 2023-06-27T12:45:22.047 INFO:tasks.rados.rados.0.smithi138.stdout:1755: writing smithi138138488-47 from 1847402 to 2201915 tid 2 2023-06-27T12:45:22.047 INFO:tasks.rados.rados.0.smithi138.stdout:1756: snap_remove snap 203 2023-06-27T12:45:22.600 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:22.671 INFO:tasks.rados.rados.0.smithi138.stdout:1756: done (5 left) 2023-06-27T12:45:22.671 INFO:tasks.rados.rados.0.smithi138.stdout:1757: read oid 28 snap -1 2023-06-27T12:45:22.671 INFO:tasks.rados.rados.0.smithi138.stdout:1757: expect (ObjNum 820 snap 199 seq_num 820) 2023-06-27T12:45:22.671 INFO:tasks.rados.rados.0.smithi138.stdout:1758: write oid 14 current snap is 204 2023-06-27T12:45:22.672 INFO:tasks.rados.rados.0.smithi138.stdout:1751: expect (ObjNum 788 snap 192 seq_num 788) 2023-06-27T12:45:22.735 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:22.730+0000 7f139dbff700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:22.738 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:22.734+0000 7f139c99d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:22.783 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:22.786 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:22.786 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:22.969 INFO:tasks.rados.rados.0.smithi138.stdout:1753: expect (ObjNum 834 snap 203 seq_num 834) 2023-06-27T12:45:23.152 INFO:tasks.rados.rados.0.smithi138.stdout:1752: finishing write tid 1 to smithi138138488-43 2023-06-27T12:45:23.152 INFO:tasks.rados.rados.0.smithi138.stdout:1755: finishing write tid 1 to smithi138138488-47 2023-06-27T12:45:23.152 INFO:tasks.rados.rados.0.smithi138.stdout:1752: finishing write tid 2 to smithi138138488-43 2023-06-27T12:45:23.152 INFO:tasks.rados.rados.0.smithi138.stdout:1758: seq_num 846 ranges {618722=575837,1905884=312838} 2023-06-27T12:45:23.158 INFO:tasks.rados.rados.0.smithi138.stdout:1758: writing smithi138138488-14 from 618722 to 1194559 tid 1 2023-06-27T12:45:23.165 INFO:tasks.rados.rados.0.smithi138.stdout:1758: writing smithi138138488-14 from 1905884 to 2218722 tid 2 2023-06-27T12:45:23.165 INFO:tasks.rados.rados.0.smithi138.stdout:1751: done (6 left) 2023-06-27T12:45:23.165 INFO:tasks.rados.rados.0.smithi138.stdout:1753: done (5 left) 2023-06-27T12:45:23.165 INFO:tasks.rados.rados.0.smithi138.stdout:1759: read oid 44 snap 202 2023-06-27T12:45:23.165 INFO:tasks.rados.rados.0.smithi138.stdout:1759: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:45:23.165 INFO:tasks.rados.rados.0.smithi138.stdout:1760: delete oid 8 current snap is 204 2023-06-27T12:45:23.165 INFO:tasks.rados.rados.0.smithi138.stdout:1755: finishing write tid 2 to smithi138138488-47 2023-06-27T12:45:23.166 INFO:tasks.rados.rados.0.smithi138.stdout:1755: finishing write tid 3 to smithi138138488-47 2023-06-27T12:45:23.166 INFO:tasks.rados.rados.0.smithi138.stdout:1755: finishing write tid 4 to smithi138138488-47 2023-06-27T12:45:23.166 INFO:tasks.rados.rados.0.smithi138.stdout:1755: oid 47 updating version 0 to 329 2023-06-27T12:45:23.166 INFO:tasks.rados.rados.0.smithi138.stdout:1755: oid 47 version 329 is already newer than 329 2023-06-27T12:45:23.166 INFO:tasks.rados.rados.0.smithi138.stdout:1755: oid 47 version 329 is already newer than 329 2023-06-27T12:45:23.166 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 47 v 329 (ObjNum 845 snap 204 seq_num 845) dirty exists 2023-06-27T12:45:23.166 INFO:tasks.rados.rados.0.smithi138.stdout:1755: left oid 47 (ObjNum 845 snap 204 seq_num 845) 2023-06-27T12:45:23.167 INFO:tasks.rados.rados.0.smithi138.stdout:1754: finishing write tid 1 to smithi138138488-24 2023-06-27T12:45:23.167 INFO:tasks.rados.rados.0.smithi138.stdout:1754: finishing write tid 2 to smithi138138488-24 2023-06-27T12:45:23.167 INFO:tasks.rados.rados.0.smithi138.stdout:1754: finishing write tid 3 to smithi138138488-24 2023-06-27T12:45:23.167 INFO:tasks.rados.rados.0.smithi138.stdout:1754: finishing write tid 4 to smithi138138488-24 2023-06-27T12:45:23.167 INFO:tasks.rados.rados.0.smithi138.stdout:1754: oid 24 updating version 0 to 473 2023-06-27T12:45:23.167 INFO:tasks.rados.rados.0.smithi138.stdout:1754: oid 24 version 473 is already newer than 473 2023-06-27T12:45:23.167 INFO:tasks.rados.rados.0.smithi138.stdout:1754: oid 24 version 473 is already newer than 473 2023-06-27T12:45:23.167 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 24 v 473 (ObjNum 844 snap 204 seq_num 844) dirty exists 2023-06-27T12:45:23.168 INFO:tasks.rados.rados.0.smithi138.stdout:1754: left oid 24 (ObjNum 844 snap 204 seq_num 844) 2023-06-27T12:45:23.168 INFO:tasks.rados.rados.0.smithi138.stdout:1752: finishing write tid 3 to smithi138138488-43 2023-06-27T12:45:23.168 INFO:tasks.rados.rados.0.smithi138.stdout:1752: finishing write tid 4 to smithi138138488-43 2023-06-27T12:45:23.168 INFO:tasks.rados.rados.0.smithi138.stdout:1752: oid 43 updating version 0 to 323 2023-06-27T12:45:23.168 INFO:tasks.rados.rados.0.smithi138.stdout:1752: oid 43 version 323 is already newer than 320 2023-06-27T12:45:23.168 INFO:tasks.rados.rados.0.smithi138.stdout:1752: oid 43 version 323 is already newer than 323 2023-06-27T12:45:23.168 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 323 (ObjNum 843 snap 204 seq_num 843) dirty exists 2023-06-27T12:45:23.169 INFO:tasks.rados.rados.0.smithi138.stdout:1752: left oid 43 (ObjNum 843 snap 204 seq_num 843) 2023-06-27T12:45:23.169 INFO:tasks.rados.rados.0.smithi138.stdout:1757: expect (ObjNum 820 snap 199 seq_num 820) 2023-06-27T12:45:23.408 INFO:tasks.rados.rados.0.smithi138.stdout:1759: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:45:23.601 INFO:tasks.rados.rados.0.smithi138.stdout:1758: finishing write tid 1 to smithi138138488-14 2023-06-27T12:45:23.601 INFO:tasks.rados.rados.0.smithi138.stdout:1758: finishing write tid 2 to smithi138138488-14 2023-06-27T12:45:23.601 INFO:tasks.rados.rados.0.smithi138.stdout:1758: finishing write tid 3 to smithi138138488-14 2023-06-27T12:45:23.601 INFO:tasks.rados.rados.0.smithi138.stdout:1758: finishing write tid 4 to smithi138138488-14 2023-06-27T12:45:23.602 INFO:tasks.rados.rados.0.smithi138.stdout:1758: oid 14 updating version 0 to 558 2023-06-27T12:45:23.602 INFO:tasks.rados.rados.0.smithi138.stdout:1758: oid 14 version 558 is already newer than 558 2023-06-27T12:45:23.602 INFO:tasks.rados.rados.0.smithi138.stdout:1758: oid 14 version 558 is already newer than 558 2023-06-27T12:45:23.602 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 14 v 558 (ObjNum 846 snap 204 seq_num 846) dirty exists 2023-06-27T12:45:23.602 INFO:tasks.rados.rados.0.smithi138.stdout:1758: left oid 14 (ObjNum 846 snap 204 seq_num 846) 2023-06-27T12:45:23.602 INFO:tasks.rados.rados.0.smithi138.stdout:1752: done (6 left) 2023-06-27T12:45:23.602 INFO:tasks.rados.rados.0.smithi138.stdout:1754: done (5 left) 2023-06-27T12:45:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:1755: done (4 left) 2023-06-27T12:45:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:1757: done (3 left) 2023-06-27T12:45:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:1758: done (2 left) 2023-06-27T12:45:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:1759: done (1 left) 2023-06-27T12:45:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:1760: done (0 left) 2023-06-27T12:45:23.603 INFO:tasks.rados.rados.0.smithi138.stdout:1761: snap_create 2023-06-27T12:45:23.673 INFO:tasks.rados.rados.0.smithi138.stdout:1761: done (0 left) 2023-06-27T12:45:23.673 INFO:tasks.rados.rados.0.smithi138.stdout:1762: snap_remove snap 204 2023-06-27T12:45:23.866 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:45:23.866 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:45:23.867 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:45:23.995 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:23.988+0000 7f6550df0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:23.998 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:23.994+0000 7f6550df0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:24.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:45:24.196 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:45:24.208 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":646,"stamp":"2023-06-27T12:45:24.012167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":119336069,"num_objects":458,"num_object_clones":411,"num_object_copies":1374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":2640,"num_read_kb":1974144,"num_write":2404,"num_write_kb":704844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4501,"ondisk_log_size":4501,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1338032,"kb_used_data":1193872,"kb_used_omap":0,"kb_used_meta":144128,"kb_avail":376149328,"statfs":{"total":386547056640,"available":385176911872,"internally_reserved":0,"allocated":1222524928,"data_stored":1431929190,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":147587072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3898894,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":13,"num_read_kb":9177,"num_write":20,"num_write_kb":4805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000841"},"pg_stats":[{"pgid":"3.f","version":"412'75","reported_seq":1015,"reported_epoch":423,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:18.677791+0000","last_change":"2023-06-27T12:45:18.804080+0000","last_active":"2023-06-27T12:45:18.677791+0000","last_peered":"2023-06-27T12:45:18.677791+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:45:18.677791+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3680898,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":26,"num_read_kb":6253,"num_write":47,"num_write_kb":13642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"426'329","reported_seq":1350,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:22.676198+0000","last_change":"2023-06-27T12:45:22.689295+0000","last_active":"2023-06-27T12:45:22.676198+0000","last_peered":"2023-06-27T12:45:22.676198+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:45:22.676198+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10084926,"num_objects":38,"num_object_clones":34,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":182,"num_read_kb":220855,"num_write":202,"num_write_kb":60670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":38}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"418'381","reported_seq":1344,"reported_epoch":423,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:18.854412+0000","last_change":"2023-06-27T12:45:18.855361+0000","last_active":"2023-06-27T12:45:18.854412+0000","last_peered":"2023-06-27T12:45:18.854412+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:45:18.854412+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16959022,"num_objects":42,"num_object_clones":37,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":321,"num_read_kb":224751,"num_write":235,"num_write_kb":69262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"402'248","reported_seq":1251,"reported_epoch":423,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:18.692538+0000","last_change":"2023-06-27T12:45:18.732908+0000","last_active":"2023-06-27T12:45:18.692538+0000","last_peered":"2023-06-27T12:45:18.692538+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:45:18.692538+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874591,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":239,"num_read_kb":179662,"num_write":156,"num_write_kb":44779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"413'115","reported_seq":752,"reported_epoch":423,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:18.678209+0000","last_change":"2023-06-27T12:45:18.791176+0000","last_active":"2023-06-27T12:45:18.678209+0000","last_peered":"2023-06-27T12:45:18.678209+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:45:18.678209+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6655869,"num_objects":10,"num_object_clones":9,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":96067,"num_write":63,"num_write_kb":19283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"409'464","reported_seq":1315,"reported_epoch":423,"state":"active+clean","last_fresh":"2023-06-27T12:45:18.672466+0000","last_change":"2023-06-27T12:45:18.693874+0000","last_active":"2023-06-27T12:45:18.672466+0000","last_peered":"2023-06-27T12:45:18.672466+0000","last_clean":"2023-06-27T12:45:18.672466+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:45:18.672466+0000","last_undegraded":"2023-06-27T12:45:18.672466+0000","last_fullsized":"2023-06-27T12:45:18.672466+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11479808,"num_objects":50,"num_object_clones":44,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":349,"num_read_kb":223353,"num_write":270,"num_write_kb":77359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"415'317","reported_seq":1160,"reported_epoch":423,"state":"active+clean","last_fresh":"2023-06-27T12:45:18.804434+0000","last_change":"2023-06-27T12:45:18.894410+0000","last_active":"2023-06-27T12:45:18.804434+0000","last_peered":"2023-06-27T12:45:18.804434+0000","last_clean":"2023-06-27T12:45:18.804434+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:45:18.804434+0000","last_undegraded":"2023-06-27T12:45:18.804434+0000","last_fullsized":"2023-06-27T12:45:18.804434+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11009080,"num_objects":36,"num_object_clones":33,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":171,"num_read_kb":108128,"num_write":182,"num_write_kb":53878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"422'101","reported_seq":1050,"reported_epoch":423,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:18.676502+0000","last_change":"2023-06-27T12:45:18.784972+0000","last_active":"2023-06-27T12:45:18.676502+0000","last_peered":"2023-06-27T12:45:18.676502+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:45:18.676502+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":11,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":69,"num_read_kb":47130,"num_write":55,"num_write_kb":16677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":722,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-06-27T12:45:22.680420+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:45:22.680420+0000","last_peered":"2023-06-27T12:45:22.680420+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:45:22.680420+0000","last_undegraded":"2023-06-27T12:45:22.680420+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":944,"reported_epoch":423,"state":"active+undersized","last_fresh":"2023-06-27T12:45:18.696088+0000","last_change":"2023-06-27T12:45:18.737227+0000","last_active":"2023-06-27T12:45:18.696088+0000","last_peered":"2023-06-27T12:45:18.696088+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:45:18.696088+0000","last_undegraded":"2023-06-27T12:45:18.696088+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":689,"reported_epoch":423,"state":"active+clean","last_fresh":"2023-06-27T12:45:18.668567+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:45:18.668567+0000","last_peered":"2023-06-27T12:45:18.668567+0000","last_clean":"2023-06-27T12:45:18.668567+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:45:18.668567+0000","last_undegraded":"2023-06-27T12:45:18.668567+0000","last_fullsized":"2023-06-27T12:45:18.668567+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"424'253","reported_seq":1230,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:22.684766+0000","last_change":"2023-06-27T12:45:22.690272+0000","last_active":"2023-06-27T12:45:22.684766+0000","last_peered":"2023-06-27T12:45:22.684766+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:45:22.684766+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6807363,"num_objects":27,"num_object_clones":22,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":105,"num_read_kb":90894,"num_write":154,"num_write_kb":42841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":27}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":682,"reported_epoch":423,"state":"active+undersized","last_fresh":"2023-06-27T12:45:18.677922+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:45:18.677922+0000","last_peered":"2023-06-27T12:45:18.677922+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:45:18.677922+0000","last_undegraded":"2023-06-27T12:45:18.677922+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"415'531","reported_seq":931,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:22.682158+0000","last_change":"2023-06-27T12:45:22.690873+0000","last_active":"2023-06-27T12:45:22.682158+0000","last_peered":"2023-06-27T12:45:22.682158+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:45:22.682158+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722932,"num_objects":34,"num_object_clones":32,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":101,"num_read_kb":81632,"num_write":172,"num_write_kb":53028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":34}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":679,"reported_epoch":423,"state":"active+clean","last_fresh":"2023-06-27T12:45:18.664927+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:45:18.664927+0000","last_peered":"2023-06-27T12:45:18.664927+0000","last_clean":"2023-06-27T12:45:18.664927+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:45:18.664927+0000","last_undegraded":"2023-06-27T12:45:18.664927+0000","last_fullsized":"2023-06-27T12:45:18.664927+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":441,"reported_epoch":423,"state":"active+undersized","last_fresh":"2023-06-27T12:45:18.685490+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:45:18.685490+0000","last_peered":"2023-06-27T12:45:18.685490+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:45:18.685490+0000","last_undegraded":"2023-06-27T12:45:18.685490+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"420'522","reported_seq":1469,"reported_epoch":423,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:18.672228+0000","last_change":"2023-06-27T12:45:18.864380+0000","last_active":"2023-06-27T12:45:18.672228+0000","last_peered":"2023-06-27T12:45:18.672228+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:45:18.672228+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18757437,"num_objects":61,"num_object_clones":55,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":61,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":361,"num_read_kb":215337,"num_write":343,"num_write_kb":101465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":61}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":689,"reported_epoch":423,"state":"active+clean","last_fresh":"2023-06-27T12:45:18.673893+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:45:18.673893+0000","last_peered":"2023-06-27T12:45:18.673893+0000","last_clean":"2023-06-27T12:45:18.673893+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:45:18.673893+0000","last_undegraded":"2023-06-27T12:45:18.673893+0000","last_fullsized":"2023-06-27T12:45:18.673893+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"420'301","reported_seq":831,"reported_epoch":423,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:18.690237+0000","last_change":"2023-06-27T12:45:18.865330+0000","last_active":"2023-06-27T12:45:18.690237+0000","last_peered":"2023-06-27T12:45:18.690237+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:45:18.690237+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3311210,"num_objects":23,"num_object_clones":23,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":177,"num_read_kb":120046,"num_write":122,"num_write_kb":34526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":23}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":465,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-06-27T12:45:22.674805+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:45:22.674805+0000","last_peered":"2023-06-27T12:45:22.674805+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:45:22.674805+0000","last_undegraded":"2023-06-27T12:45:22.674805+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"426'323","reported_seq":1305,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:22.684651+0000","last_change":"2023-06-27T12:45:22.756571+0000","last_active":"2023-06-27T12:45:22.684651+0000","last_peered":"2023-06-27T12:45:22.684651+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:45:22.684651+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8051114,"num_objects":39,"num_object_clones":35,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":149,"num_read_kb":155688,"num_write":196,"num_write_kb":56712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":39}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"421'185","reported_seq":760,"reported_epoch":423,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:18.688085+0000","last_change":"2023-06-27T12:45:18.803726+0000","last_active":"2023-06-27T12:45:18.688085+0000","last_peered":"2023-06-27T12:45:18.688085+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:45:18.688085+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3933003,"num_objects":14,"num_object_clones":14,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":121,"num_read_kb":68415,"num_write":72,"num_write_kb":20805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":722,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-06-27T12:45:22.678780+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:45:22.678780+0000","last_peered":"2023-06-27T12:45:22.678780+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:45:22.678780+0000","last_undegraded":"2023-06-27T12:45:22.678780+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":679,"reported_epoch":423,"state":"active+clean","last_fresh":"2023-06-27T12:45:18.671951+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:45:18.671951+0000","last_peered":"2023-06-27T12:45:18.671951+0000","last_clean":"2023-06-27T12:45:18.671951+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:45:18.671951+0000","last_undegraded":"2023-06-27T12:45:18.671951+0000","last_fullsized":"2023-06-27T12:45:18.671951+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"412'259","reported_seq":1240,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:22.681634+0000","last_change":"2023-06-27T12:45:22.687018+0000","last_active":"2023-06-27T12:45:22.681634+0000","last_peered":"2023-06-27T12:45:22.681634+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:45:22.681634+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7008816,"num_objects":28,"num_object_clones":25,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":128,"num_read_kb":135933,"num_write":135,"num_write_kb":39917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":119336069,"num_objects":458,"num_object_clones":411,"num_object_copies":1374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":2640,"num_read_kb":1974144,"num_write":2404,"num_write_kb":704844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4404,"ondisk_log_size":4404,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673410,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":361364,"kb_used_data":326348,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94010476,"statfs":{"total":96636764160,"available":96266727424,"internally_reserved":0,"allocated":334180352,"data_stored":393382810,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706110,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":385228,"kb_used_data":346116,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93986612,"statfs":{"total":96636764160,"available":96242290688,"internally_reserved":0,"allocated":354422784,"data_stored":416289549,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673405,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":410160,"kb_used_data":371048,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93961680,"statfs":{"total":96636764160,"available":96216760320,"internally_reserved":0,"allocated":379953152,"data_stored":445217279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:45:24.210 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.e 3.d 3.b 3.8 3.0 3.2 3.1 3.4 3.7 2023-06-27T12:45:24.349 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:24.331+0000 7f459547e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:24.352 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:24.348+0000 7f459547e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:24.544 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b,3.f] on osd.0 to force-backfill; instructing pg(s) [3.1,3.2,3.7,3.8,3.d] on osd.1 to force-backfill; instructing pg(s) [3.0,3.4,3.e] on osd.2 to force-backfill; 2023-06-27T12:45:24.545 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:45:24.672 INFO:tasks.rados.rados.0.smithi138.stdout:1762: done (0 left) 2023-06-27T12:45:24.673 INFO:tasks.rados.rados.0.smithi138.stdout:1763: snap_remove snap 202 2023-06-27T12:45:24.787 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:45:24.913 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:24.909+0000 7f63ebc60700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:24.916 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:24.912+0000 7f63ebc60700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:24.965 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:24.967 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:24.968 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:25.479 INFO:tasks.rados.rados.0.smithi138.stdout:1763: done (0 left) 2023-06-27T12:45:25.479 INFO:tasks.rados.rados.0.smithi138.stdout:1764: read oid 47 snap -1 2023-06-27T12:45:25.480 INFO:tasks.rados.rados.0.smithi138.stdout:1764: expect (ObjNum 845 snap 204 seq_num 845) 2023-06-27T12:45:25.480 INFO:tasks.rados.rados.0.smithi138.stdout:1765: delete oid 25 current snap is 205 2023-06-27T12:45:25.532 INFO:tasks.rados.rados.0.smithi138.stdout:1764: expect (ObjNum 845 snap 204 seq_num 845) 2023-06-27T12:45:25.637 INFO:tasks.rados.rados.0.smithi138.stdout:1764: done (1 left) 2023-06-27T12:45:25.637 INFO:tasks.rados.rados.0.smithi138.stdout:1765: done (0 left) 2023-06-27T12:45:25.637 INFO:tasks.rados.rados.0.smithi138.stdout:1766: snap_remove snap 201 2023-06-27T12:45:26.487 INFO:tasks.rados.rados.0.smithi138.stdout:1766: done (0 left) 2023-06-27T12:45:26.487 INFO:tasks.rados.rados.0.smithi138.stdout:1767: delete oid 18 current snap is 205 2023-06-27T12:45:26.548 INFO:tasks.rados.rados.0.smithi138.stdout:1767: done (0 left) 2023-06-27T12:45:26.549 INFO:tasks.rados.rados.0.smithi138.stdout:1768: snap_remove snap 175 2023-06-27T12:45:26.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:27.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~292s 2023-06-27T12:45:27.103 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:27.098+0000 7fb737620700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:27.106 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:27.102+0000 7fb737620700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:27.152 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:27.155 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:27.155 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:27.489 INFO:tasks.rados.rados.0.smithi138.stdout:1768: done (0 left) 2023-06-27T12:45:27.490 INFO:tasks.rados.rados.0.smithi138.stdout:1769: write (excl) oid 30 current snap is 205 2023-06-27T12:45:27.490 INFO:tasks.rados.rados.0.smithi138.stdout:1769: seq_num 850 ranges {685950=476833,1623459=570617} 2023-06-27T12:45:27.495 INFO:tasks.rados.rados.0.smithi138.stdout:1769: writing smithi138138488-30 from 685950 to 1162783 tid 1 2023-06-27T12:45:27.502 INFO:tasks.rados.rados.0.smithi138.stdout:1769: writing smithi138138488-30 from 1623459 to 2194076 tid 2 2023-06-27T12:45:27.503 INFO:tasks.rados.rados.0.smithi138.stdout:1770: write (excl) oid 29 current snap is 205 2023-06-27T12:45:27.503 INFO:tasks.rados.rados.0.smithi138.stdout:1770: seq_num 851 ranges {702757=552082,1681941=430628} 2023-06-27T12:45:27.511 INFO:tasks.rados.rados.0.smithi138.stdout:1770: writing smithi138138488-29 from 702757 to 1254839 tid 1 2023-06-27T12:45:27.518 INFO:tasks.rados.rados.0.smithi138.stdout:1770: writing smithi138138488-29 from 1681941 to 2112569 tid 2 2023-06-27T12:45:27.518 INFO:tasks.rados.rados.0.smithi138.stdout:1771: read oid 45 snap -1 2023-06-27T12:45:27.518 INFO:tasks.rados.rados.0.smithi138.stdout:1771: expect (ObjNum 764 snap 189 seq_num 764) 2023-06-27T12:45:27.518 INFO:tasks.rados.rados.0.smithi138.stdout:1772: delete oid 37 current snap is 205 2023-06-27T12:45:27.556 INFO:tasks.rados.rados.0.smithi138.stdout:1769: finishing write tid 1 to smithi138138488-30 2023-06-27T12:45:27.568 INFO:tasks.rados.rados.0.smithi138.stdout:1771: expect (ObjNum 764 snap 189 seq_num 764) 2023-06-27T12:45:27.683 INFO:tasks.rados.rados.0.smithi138.stdout:1769: finishing write tid 2 to smithi138138488-30 2023-06-27T12:45:27.683 INFO:tasks.rados.rados.0.smithi138.stdout:1770: finishing write tid 1 to smithi138138488-29 2023-06-27T12:45:27.684 INFO:tasks.rados.rados.0.smithi138.stdout:1769: finishing write tid 3 to smithi138138488-30 2023-06-27T12:45:27.684 INFO:tasks.rados.rados.0.smithi138.stdout:1769: finishing write tid 4 to smithi138138488-30 2023-06-27T12:45:27.684 INFO:tasks.rados.rados.0.smithi138.stdout:1769: oid 30 updating version 0 to 319 2023-06-27T12:45:27.684 INFO:tasks.rados.rados.0.smithi138.stdout:1769: oid 30 version 319 is already newer than 317 2023-06-27T12:45:27.684 INFO:tasks.rados.rados.0.smithi138.stdout:1769: oid 30 version 319 is already newer than 318 2023-06-27T12:45:27.684 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 30 v 319 (ObjNum 850 snap 205 seq_num 850) dirty exists 2023-06-27T12:45:27.684 INFO:tasks.rados.rados.0.smithi138.stdout:1769: left oid 30 (ObjNum 850 snap 205 seq_num 850) 2023-06-27T12:45:27.685 INFO:tasks.rados.rados.0.smithi138.stdout:1770: finishing write tid 2 to smithi138138488-29 2023-06-27T12:45:27.685 INFO:tasks.rados.rados.0.smithi138.stdout:1769: done (3 left) 2023-06-27T12:45:27.685 INFO:tasks.rados.rados.0.smithi138.stdout:1771: done (2 left) 2023-06-27T12:45:27.685 INFO:tasks.rados.rados.0.smithi138.stdout:1772: done (1 left) 2023-06-27T12:45:27.685 INFO:tasks.rados.rados.0.smithi138.stdout:1773: write (excl) oid 22 current snap is 205 2023-06-27T12:45:27.685 INFO:tasks.rados.rados.0.smithi138.stdout:1770: finishing write tid 3 to smithi138138488-29 2023-06-27T12:45:27.685 INFO:tasks.rados.rados.0.smithi138.stdout:1770: finishing write tid 4 to smithi138138488-29 2023-06-27T12:45:27.685 INFO:tasks.rados.rados.0.smithi138.stdout:1770: oid 29 updating version 0 to 195 2023-06-27T12:45:27.686 INFO:tasks.rados.rados.0.smithi138.stdout:1770: oid 29 updating version 195 to 197 2023-06-27T12:45:27.686 INFO:tasks.rados.rados.0.smithi138.stdout:1770: oid 29 version 197 is already newer than 197 2023-06-27T12:45:27.686 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 29 v 197 (ObjNum 851 snap 205 seq_num 851) dirty exists 2023-06-27T12:45:27.686 INFO:tasks.rados.rados.0.smithi138.stdout:1770: left oid 29 (ObjNum 851 snap 205 seq_num 851) 2023-06-27T12:45:27.686 INFO:tasks.rados.rados.0.smithi138.stdout:1773: seq_num 853 ranges {736371=418933,1798905=434297} 2023-06-27T12:45:27.689 INFO:tasks.rados.rados.0.smithi138.stdout:1773: writing smithi138138488-22 from 736371 to 1155304 tid 1 2023-06-27T12:45:27.695 INFO:tasks.rados.rados.0.smithi138.stdout:1773: writing smithi138138488-22 from 1798905 to 2233202 tid 2 2023-06-27T12:45:27.695 INFO:tasks.rados.rados.0.smithi138.stdout:1770: done (1 left) 2023-06-27T12:45:27.696 INFO:tasks.rados.rados.0.smithi138.stdout:1774: read oid 19 snap 198 2023-06-27T12:45:27.696 INFO:tasks.rados.rados.0.smithi138.stdout:1774: expect (ObjNum 774 snap 190 seq_num 774) 2023-06-27T12:45:27.696 INFO:tasks.rados.rados.0.smithi138.stdout:1775: read oid 44 snap -1 2023-06-27T12:45:27.696 INFO:tasks.rados.rados.0.smithi138.stdout:1775: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:45:27.696 INFO:tasks.rados.rados.0.smithi138.stdout:1776: snap_create 2023-06-27T12:45:27.732 INFO:tasks.rados.rados.0.smithi138.stdout:1774: expect (ObjNum 774 snap 190 seq_num 774) 2023-06-27T12:45:27.891 INFO:tasks.rados.rados.0.smithi138.stdout:1773: finishing write tid 1 to smithi138138488-22 2023-06-27T12:45:27.891 INFO:tasks.rados.rados.0.smithi138.stdout:1775: expect (ObjNum 695 snap 169 seq_num 695) 2023-06-27T12:45:28.084 INFO:tasks.rados.rados.0.smithi138.stdout:1773: finishing write tid 2 to smithi138138488-22 2023-06-27T12:45:28.084 INFO:tasks.rados.rados.0.smithi138.stdout:1773: finishing write tid 3 to smithi138138488-22 2023-06-27T12:45:28.084 INFO:tasks.rados.rados.0.smithi138.stdout:1773: finishing write tid 4 to smithi138138488-22 2023-06-27T12:45:28.084 INFO:tasks.rados.rados.0.smithi138.stdout:1773: oid 22 updating version 0 to 269 2023-06-27T12:45:28.085 INFO:tasks.rados.rados.0.smithi138.stdout:1773: oid 22 version 269 is already newer than 269 2023-06-27T12:45:28.085 INFO:tasks.rados.rados.0.smithi138.stdout:1773: oid 22 version 269 is already newer than 266 2023-06-27T12:45:28.085 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 22 v 269 (ObjNum 853 snap 205 seq_num 853) dirty exists 2023-06-27T12:45:28.085 INFO:tasks.rados.rados.0.smithi138.stdout:1773: left oid 22 (ObjNum 853 snap 205 seq_num 853) 2023-06-27T12:45:28.503 INFO:tasks.rados.rados.0.smithi138.stdout:1773: done (3 left) 2023-06-27T12:45:28.503 INFO:tasks.rados.rados.0.smithi138.stdout:1774: done (2 left) 2023-06-27T12:45:28.504 INFO:tasks.rados.rados.0.smithi138.stdout:1775: done (1 left) 2023-06-27T12:45:28.504 INFO:tasks.rados.rados.0.smithi138.stdout:1776: done (0 left) 2023-06-27T12:45:28.504 INFO:tasks.rados.rados.0.smithi138.stdout:1777: read oid 35 snap -1 2023-06-27T12:45:28.505 INFO:tasks.rados.rados.0.smithi138.stdout:1777: expect (ObjNum 788 snap 192 seq_num 788) 2023-06-27T12:45:28.505 INFO:tasks.rados.rados.0.smithi138.stdout:1778: read oid 5 snap -1 2023-06-27T12:45:28.505 INFO:tasks.rados.rados.0.smithi138.stdout:1778: expect deleted 2023-06-27T12:45:28.505 INFO:tasks.rados.rados.0.smithi138.stdout:1779: delete oid 11 current snap is 206 2023-06-27T12:45:28.545 INFO:tasks.rados.rados.0.smithi138.stdout:1777: expect (ObjNum 788 snap 192 seq_num 788) 2023-06-27T12:45:28.822 INFO:tasks.rados.rados.0.smithi138.stdout:1777: done (2 left) 2023-06-27T12:45:28.822 INFO:tasks.rados.rados.0.smithi138.stdout:1778: done (1 left) 2023-06-27T12:45:28.823 INFO:tasks.rados.rados.0.smithi138.stdout:1779: done (0 left) 2023-06-27T12:45:28.823 INFO:tasks.rados.rados.0.smithi138.stdout:1780: delete oid 20 current snap is 206 2023-06-27T12:45:28.873 INFO:tasks.rados.rados.0.smithi138.stdout:1780: done (0 left) 2023-06-27T12:45:28.873 INFO:tasks.rados.rados.0.smithi138.stdout:1781: write (excl) oid 20 current snap is 206 2023-06-27T12:45:28.873 INFO:tasks.rados.rados.0.smithi138.stdout:1781: seq_num 856 ranges {786792=644680,1857998=528794} 2023-06-27T12:45:28.880 INFO:tasks.rados.rados.0.smithi138.stdout:1781: writing smithi138138488-20 from 786792 to 1431472 tid 1 2023-06-27T12:45:28.886 INFO:tasks.rados.rados.0.smithi138.stdout:1781: writing smithi138138488-20 from 1857998 to 2386792 tid 2 2023-06-27T12:45:28.886 INFO:tasks.rados.rados.0.smithi138.stdout:1782: read oid 26 snap -1 2023-06-27T12:45:28.886 INFO:tasks.rados.rados.0.smithi138.stdout:1782: expect (ObjNum 822 snap 200 seq_num 822) 2023-06-27T12:45:28.886 INFO:tasks.rados.rados.0.smithi138.stdout:1783: read oid 34 snap -1 2023-06-27T12:45:28.886 INFO:tasks.rados.rados.0.smithi138.stdout:1783: expect (ObjNum 724 snap 178 seq_num 724) 2023-06-27T12:45:28.887 INFO:tasks.rados.rados.0.smithi138.stdout:1784: read oid 21 snap -1 2023-06-27T12:45:28.887 INFO:tasks.rados.rados.0.smithi138.stdout:1784: expect (ObjNum 824 snap 201 seq_num 824) 2023-06-27T12:45:28.887 INFO:tasks.rados.rados.0.smithi138.stdout:1785: snap_remove snap 205 2023-06-27T12:45:29.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:45:29.282 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:29.277+0000 7f0af3793700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:29.285 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:29.281+0000 7f0af3793700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:29.330 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:29.332 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:29.333 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:29.510 INFO:tasks.rados.rados.0.smithi138.stdout:1785: done (4 left) 2023-06-27T12:45:29.510 INFO:tasks.rados.rados.0.smithi138.stdout:1786: write (excl) oid 37 current snap is 206 2023-06-27T12:45:29.510 INFO:tasks.rados.rados.0.smithi138.stdout:1786: seq_num 857 ranges {403599=719929,1916480=441635} 2023-06-27T12:45:29.518 INFO:tasks.rados.rados.0.smithi138.stdout:1786: writing smithi138138488-37 from 403599 to 1123528 tid 1 2023-06-27T12:45:29.527 INFO:tasks.rados.rados.0.smithi138.stdout:1786: writing smithi138138488-37 from 1916480 to 2358115 tid 2 2023-06-27T12:45:29.527 INFO:tasks.rados.rados.0.smithi138.stdout:1787: write (excl) oid 44 current snap is 206 2023-06-27T12:45:29.527 INFO:tasks.rados.rados.0.smithi138.stdout:1787: seq_num 858 ranges {420406=795178,1974962=445444} 2023-06-27T12:45:29.537 INFO:tasks.rados.rados.0.smithi138.stdout:1787: writing smithi138138488-44 from 420406 to 1215584 tid 1 2023-06-27T12:45:29.544 INFO:tasks.rados.rados.0.smithi138.stdout:1787: writing smithi138138488-44 from 1974962 to 2420406 tid 2 2023-06-27T12:45:29.544 INFO:tasks.rados.rados.0.smithi138.stdout:1788: delete oid 18 current snap is 206 2023-06-27T12:45:29.544 INFO:tasks.rados.rados.0.smithi138.stdout:1783: expect (ObjNum 724 snap 178 seq_num 724) 2023-06-27T12:45:29.557 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:45:29.557 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:45:29.557 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:45:29.637 INFO:tasks.rados.rados.0.smithi138.stdout:1782: expect (ObjNum 822 snap 200 seq_num 822) 2023-06-27T12:45:29.700 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:29.692+0000 7f40f1959700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:29.702 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:29.699+0000 7f40f1959700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:29.880 INFO:tasks.rados.rados.0.smithi138.stdout:1784: expect (ObjNum 824 snap 201 seq_num 824) 2023-06-27T12:45:29.901 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:45:29.902 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:45:29.914 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":654,"stamp":"2023-06-27T12:45:29.504965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":92562056,"num_objects":470,"num_object_clones":423,"num_object_copies":1410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":2719,"num_read_kb":2044543,"num_write":2449,"num_write_kb":715912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4646,"ondisk_log_size":4646,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1337984,"kb_used_data":1193164,"kb_used_omap":0,"kb_used_meta":144128,"kb_avail":376149376,"statfs":{"total":386547056640,"available":385176961024,"internally_reserved":0,"allocated":1221799936,"data_stored":1424033506,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":147587072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12420703,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":47,"num_read_kb":46720,"num_write":16,"num_write_kb":3967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.032125"},"pg_stats":[{"pgid":"3.f","version":"424'83","reported_seq":1050,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.831830+0000","last_change":"2023-06-27T12:45:27.719867+0000","last_active":"2023-06-27T12:45:28.831830+0000","last_peered":"2023-06-27T12:45:28.831830+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:45:28.831830+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2050652,"num_objects":11,"num_object_clones":8,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":28,"num_read_kb":6254,"num_write":51,"num_write_kb":14737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"433'336","reported_seq":1373,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.933420+0000","last_change":"2023-06-27T12:45:28.534577+0000","last_active":"2023-06-27T12:45:28.933420+0000","last_peered":"2023-06-27T12:45:28.933420+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:45:28.933420+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10454680,"num_objects":39,"num_object_clones":35,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":223843,"num_write":206,"num_write_kb":61817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":39}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"432'395","reported_seq":1380,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.516508+0000","last_change":"2023-06-27T12:45:27.539332+0000","last_active":"2023-06-27T12:45:28.516508+0000","last_peered":"2023-06-27T12:45:28.516508+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:45:28.516508+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3496021,"num_objects":44,"num_object_clones":39,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":323,"num_read_kb":224751,"num_write":235,"num_write_kb":69262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":44}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"402'248","reported_seq":1282,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.832493+0000","last_change":"2023-06-27T12:45:27.751794+0000","last_active":"2023-06-27T12:45:28.832493+0000","last_peered":"2023-06-27T12:45:28.832493+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:45:28.832493+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874591,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":246,"num_read_kb":188083,"num_write":156,"num_write_kb":44779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"427'134","reported_seq":791,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.835504+0000","last_change":"2023-06-27T12:45:27.720895+0000","last_active":"2023-06-27T12:45:28.835504+0000","last_peered":"2023-06-27T12:45:28.835504+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:45:28.835504+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":12,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":143,"num_read_kb":96068,"num_write":68,"num_write_kb":20943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":13}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"426'473","reported_seq":1357,"reported_epoch":433,"state":"active+clean","last_fresh":"2023-06-27T12:45:28.896782+0000","last_change":"2023-06-27T12:45:27.723399+0000","last_active":"2023-06-27T12:45:28.896782+0000","last_peered":"2023-06-27T12:45:28.896782+0000","last_clean":"2023-06-27T12:45:28.896782+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:45:28.896782+0000","last_undegraded":"2023-06-27T12:45:28.896782+0000","last_fullsized":"2023-06-27T12:45:28.896782+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13664916,"num_objects":50,"num_object_clones":44,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":371,"num_read_kb":241464,"num_write":274,"num_write_kb":78162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"425'329","reported_seq":1196,"reported_epoch":433,"state":"active+clean","last_fresh":"2023-06-27T12:45:28.836961+0000","last_change":"2023-06-27T12:45:27.753039+0000","last_active":"2023-06-27T12:45:28.836961+0000","last_peered":"2023-06-27T12:45:28.836961+0000","last_clean":"2023-06-27T12:45:28.836961+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:45:28.836961+0000","last_undegraded":"2023-06-27T12:45:28.836961+0000","last_fullsized":"2023-06-27T12:45:28.836961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7294568,"num_objects":37,"num_object_clones":34,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":178,"num_read_kb":116264,"num_write":182,"num_write_kb":53878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"422'101","reported_seq":1080,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.516400+0000","last_change":"2023-06-27T12:45:27.519113+0000","last_active":"2023-06-27T12:45:28.516400+0000","last_peered":"2023-06-27T12:45:28.516400+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:45:28.516400+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":11,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":69,"num_read_kb":47130,"num_write":55,"num_write_kb":16677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":732,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-06-27T12:45:28.516905+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:45:28.516905+0000","last_peered":"2023-06-27T12:45:28.516905+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:45:28.516905+0000","last_undegraded":"2023-06-27T12:45:28.516905+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":971,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-06-27T12:45:28.514838+0000","last_change":"2023-06-27T12:45:27.519608+0000","last_active":"2023-06-27T12:45:28.514838+0000","last_peered":"2023-06-27T12:45:28.514838+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:45:28.514838+0000","last_undegraded":"2023-06-27T12:45:28.514838+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":709,"reported_epoch":433,"state":"active+clean","last_fresh":"2023-06-27T12:45:28.834387+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:45:28.834387+0000","last_peered":"2023-06-27T12:45:28.834387+0000","last_clean":"2023-06-27T12:45:28.834387+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:45:28.834387+0000","last_undegraded":"2023-06-27T12:45:28.834387+0000","last_fullsized":"2023-06-27T12:45:28.834387+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"424'253","reported_seq":1246,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.531522+0000","last_change":"2023-06-27T12:45:28.532836+0000","last_active":"2023-06-27T12:45:28.531522+0000","last_peered":"2023-06-27T12:45:28.531522+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:45:28.531522+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6807363,"num_objects":27,"num_object_clones":22,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":105,"num_read_kb":90894,"num_write":154,"num_write_kb":42841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":27}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":702,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-06-27T12:45:28.511800+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:45:28.511800+0000","last_peered":"2023-06-27T12:45:28.511800+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:45:28.511800+0000","last_undegraded":"2023-06-27T12:45:28.511800+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"415'531","reported_seq":950,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.535471+0000","last_change":"2023-06-27T12:45:28.529626+0000","last_active":"2023-06-27T12:45:28.535471+0000","last_peered":"2023-06-27T12:45:28.535471+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:45:28.535471+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2722932,"num_objects":34,"num_object_clones":32,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":105,"num_read_kb":92938,"num_write":172,"num_write_kb":53028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":34}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":699,"reported_epoch":433,"state":"active+clean","last_fresh":"2023-06-27T12:45:28.514779+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:45:28.514779+0000","last_peered":"2023-06-27T12:45:28.514779+0000","last_clean":"2023-06-27T12:45:28.514779+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:45:28.514779+0000","last_undegraded":"2023-06-27T12:45:28.514779+0000","last_fullsized":"2023-06-27T12:45:28.514779+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":461,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-06-27T12:45:28.843521+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:45:28.843521+0000","last_peered":"2023-06-27T12:45:28.843521+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:45:28.843521+0000","last_undegraded":"2023-06-27T12:45:28.843521+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"427'558","reported_seq":1519,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.901734+0000","last_change":"2023-06-27T12:45:27.514868+0000","last_active":"2023-06-27T12:45:28.901734+0000","last_peered":"2023-06-27T12:45:28.901734+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:45:28.901734+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15698629,"num_objects":64,"num_object_clones":58,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":379,"num_read_kb":232114,"num_write":355,"num_write_kb":104092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":64}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":709,"reported_epoch":433,"state":"active+clean","last_fresh":"2023-06-27T12:45:28.836364+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:45:28.836364+0000","last_peered":"2023-06-27T12:45:28.836364+0000","last_clean":"2023-06-27T12:45:28.836364+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:45:28.836364+0000","last_undegraded":"2023-06-27T12:45:28.836364+0000","last_fullsized":"2023-06-27T12:45:28.836364+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"432'319","reported_seq":868,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.509821+0000","last_change":"2023-06-27T12:45:27.583189+0000","last_active":"2023-06-27T12:45:28.509821+0000","last_peered":"2023-06-27T12:45:28.509821+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:45:28.509821+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5597160,"num_objects":23,"num_object_clones":23,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":180,"num_read_kb":120048,"num_write":130,"num_write_kb":36466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":23}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":475,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-06-27T12:45:28.516726+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:45:28.516726+0000","last_peered":"2023-06-27T12:45:28.516726+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:45:28.516726+0000","last_undegraded":"2023-06-27T12:45:28.516726+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"426'323","reported_seq":1324,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.896592+0000","last_change":"2023-06-27T12:45:28.523897+0000","last_active":"2023-06-27T12:45:28.896592+0000","last_peered":"2023-06-27T12:45:28.896592+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:45:28.896592+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8051114,"num_objects":39,"num_object_clones":35,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":151,"num_read_kb":160342,"num_write":196,"num_write_kb":56712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":39}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"432'197","reported_seq":795,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.515403+0000","last_change":"2023-06-27T12:45:27.513052+0000","last_active":"2023-06-27T12:45:28.515403+0000","last_peered":"2023-06-27T12:45:28.515403+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:45:28.515403+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2302757,"num_objects":14,"num_object_clones":14,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":124,"num_read_kb":68416,"num_write":76,"num_write_kb":21766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":732,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-06-27T12:45:28.518487+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:45:28.518487+0000","last_peered":"2023-06-27T12:45:28.518487+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:45:28.518487+0000","last_undegraded":"2023-06-27T12:45:28.518487+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":699,"reported_epoch":433,"state":"active+clean","last_fresh":"2023-06-27T12:45:28.512250+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:45:28.512250+0000","last_peered":"2023-06-27T12:45:28.512250+0000","last_clean":"2023-06-27T12:45:28.512250+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:45:28.512250+0000","last_undegraded":"2023-06-27T12:45:28.512250+0000","last_fullsized":"2023-06-27T12:45:28.512250+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"432'269","reported_seq":1259,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:28.512562+0000","last_change":"2023-06-27T12:45:28.521455+0000","last_active":"2023-06-27T12:45:28.512562+0000","last_peered":"2023-06-27T12:45:28.512562+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:45:28.512562+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5546673,"num_objects":29,"num_object_clones":26,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":130,"num_read_kb":135934,"num_write":139,"num_write_kb":40752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":29}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":92562056,"num_objects":470,"num_object_clones":423,"num_object_copies":1410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":2719,"num_read_kb":2044543,"num_write":2449,"num_write_kb":715912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4549,"ondisk_log_size":4549,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673415,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":362396,"kb_used_data":327380,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94009444,"statfs":{"total":96636764160,"available":96265670656,"internally_reserved":0,"allocated":335237120,"data_stored":394476197,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706120,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":384248,"kb_used_data":345136,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93987592,"statfs":{"total":96636764160,"available":96243294208,"internally_reserved":0,"allocated":353419264,"data_stored":412076756,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673415,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":410060,"kb_used_data":370288,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93961780,"statfs":{"total":96636764160,"available":96216862720,"internally_reserved":0,"allocated":379174912,"data_stored":440441001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:45:29.916 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.e 3.b 3.0 3.4 3.7 2023-06-27T12:45:30.031 INFO:tasks.rados.rados.0.smithi138.stdout:1781: finishing write tid 1 to smithi138138488-20 2023-06-27T12:45:30.032 INFO:tasks.rados.rados.0.smithi138.stdout:1781: finishing write tid 2 to smithi138138488-20 2023-06-27T12:45:30.032 INFO:tasks.rados.rados.0.smithi138.stdout:1781: finishing write tid 3 to smithi138138488-20 2023-06-27T12:45:30.032 INFO:tasks.rados.rados.0.smithi138.stdout:1781: finishing write tid 4 to smithi138138488-20 2023-06-27T12:45:30.033 INFO:tasks.rados.rados.0.smithi138.stdout:1781: oid 20 updating version 0 to 336 2023-06-27T12:45:30.033 INFO:tasks.rados.rados.0.smithi138.stdout:1781: oid 20 version 336 is already newer than 334 2023-06-27T12:45:30.033 INFO:tasks.rados.rados.0.smithi138.stdout:1781: oid 20 version 336 is already newer than 336 2023-06-27T12:45:30.033 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 20 v 336 (ObjNum 856 snap 206 seq_num 856) dirty exists 2023-06-27T12:45:30.033 INFO:tasks.rados.rados.0.smithi138.stdout:1781: left oid 20 (ObjNum 856 snap 206 seq_num 856) 2023-06-27T12:45:30.034 INFO:tasks.rados.rados.0.smithi138.stdout:1786: finishing write tid 1 to smithi138138488-37 2023-06-27T12:45:30.034 INFO:tasks.rados.rados.0.smithi138.stdout:1781: done (6 left) 2023-06-27T12:45:30.034 INFO:tasks.rados.rados.0.smithi138.stdout:1782: done (5 left) 2023-06-27T12:45:30.034 INFO:tasks.rados.rados.0.smithi138.stdout:1783: done (4 left) 2023-06-27T12:45:30.034 INFO:tasks.rados.rados.0.smithi138.stdout:1784: done (3 left) 2023-06-27T12:45:30.034 INFO:tasks.rados.rados.0.smithi138.stdout:1788: done (2 left) 2023-06-27T12:45:30.035 INFO:tasks.rados.rados.0.smithi138.stdout:1789: delete oid 11 current snap is 206 2023-06-27T12:45:30.035 INFO:tasks.rados.rados.0.smithi138.stdout:1787: finishing write tid 1 to smithi138138488-44 2023-06-27T12:45:30.035 INFO:tasks.rados.rados.0.smithi138.stdout:1786: finishing write tid 2 to smithi138138488-37 2023-06-27T12:45:30.035 INFO:tasks.rados.rados.0.smithi138.stdout:1786: finishing write tid 3 to smithi138138488-37 2023-06-27T12:45:30.035 INFO:tasks.rados.rados.0.smithi138.stdout:1786: finishing write tid 4 to smithi138138488-37 2023-06-27T12:45:30.035 INFO:tasks.rados.rados.0.smithi138.stdout:1786: oid 37 updating version 0 to 402 2023-06-27T12:45:30.035 INFO:tasks.rados.rados.0.smithi138.stdout:1786: oid 37 version 402 is already newer than 402 2023-06-27T12:45:30.036 INFO:tasks.rados.rados.0.smithi138.stdout:1786: oid 37 version 402 is already newer than 400 2023-06-27T12:45:30.036 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 37 v 402 (ObjNum 857 snap 206 seq_num 857) dirty exists 2023-06-27T12:45:30.036 INFO:tasks.rados.rados.0.smithi138.stdout:1786: left oid 37 (ObjNum 857 snap 206 seq_num 857) 2023-06-27T12:45:30.036 INFO:tasks.rados.rados.0.smithi138.stdout:1787: finishing write tid 2 to smithi138138488-44 2023-06-27T12:45:30.036 INFO:tasks.rados.rados.0.smithi138.stdout:1787: finishing write tid 3 to smithi138138488-44 2023-06-27T12:45:30.036 INFO:tasks.rados.rados.0.smithi138.stdout:1787: finishing write tid 4 to smithi138138488-44 2023-06-27T12:45:30.036 INFO:tasks.rados.rados.0.smithi138.stdout:1787: oid 44 updating version 0 to 474 2023-06-27T12:45:30.037 INFO:tasks.rados.rados.0.smithi138.stdout:1787: oid 44 updating version 474 to 477 2023-06-27T12:45:30.037 INFO:tasks.rados.rados.0.smithi138.stdout:1787: oid 44 version 477 is already newer than 477 2023-06-27T12:45:30.037 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 477 (ObjNum 858 snap 206 seq_num 858) dirty exists 2023-06-27T12:45:30.037 INFO:tasks.rados.rados.0.smithi138.stdout:1787: left oid 44 (ObjNum 858 snap 206 seq_num 858) 2023-06-27T12:45:30.038 INFO:tasks.rados.rados.0.smithi138.stdout:1786: done (2 left) 2023-06-27T12:45:30.038 INFO:tasks.rados.rados.0.smithi138.stdout:1787: done (1 left) 2023-06-27T12:45:30.038 INFO:tasks.rados.rados.0.smithi138.stdout:1789: done (0 left) 2023-06-27T12:45:30.038 INFO:tasks.rados.rados.0.smithi138.stdout:1790: read oid 39 snap -1 2023-06-27T12:45:30.038 INFO:tasks.rados.rados.0.smithi138.stdout:1790: expect (ObjNum 769 snap 189 seq_num 769) 2023-06-27T12:45:30.039 INFO:tasks.rados.rados.0.smithi138.stdout:1791: read oid 13 snap -1 2023-06-27T12:45:30.039 INFO:tasks.rados.rados.0.smithi138.stdout:1791: expect deleted 2023-06-27T12:45:30.039 INFO:tasks.rados.rados.0.smithi138.stdout:1792: write oid 36 current snap is 206 2023-06-27T12:45:30.039 INFO:tasks.rados.rados.0.smithi138.stdout:1792: seq_num 861 ranges {470827=737278,1750408=720419} 2023-06-27T12:45:30.041 INFO:tasks.rados.rados.0.smithi138.stdout:1792: writing smithi138138488-36 from 470827 to 1208105 tid 1 2023-06-27T12:45:30.049 INFO:tasks.rados.rados.0.smithi138.stdout:1792: writing smithi138138488-36 from 1750408 to 2470827 tid 2 2023-06-27T12:45:30.049 INFO:tasks.rados.rados.0.smithi138.stdout:1793: write (excl) oid 43 current snap is 206 2023-06-27T12:45:30.049 INFO:tasks.rados.rados.0.smithi138.stdout:1793: seq_num 862 ranges {487634=412527,1408890=592631} 2023-06-27T12:45:30.054 INFO:tasks.rados.rados.0.smithi138.stdout:1793: writing smithi138138488-43 from 487634 to 900161 tid 1 2023-06-27T12:45:30.057 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:30.042+0000 7f8a833ce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:30.061 INFO:tasks.rados.rados.0.smithi138.stdout:1793: writing smithi138138488-43 from 1408890 to 2001521 tid 2 2023-06-27T12:45:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:30.057+0000 7f8a833ce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:30.062 INFO:tasks.rados.rados.0.smithi138.stdout:1794: write oid 47 current snap is 206 2023-06-27T12:45:30.062 INFO:tasks.rados.rados.0.smithi138.stdout:1794: seq_num 863 ranges {504441=487776,1467372=736289} 2023-06-27T12:45:30.067 INFO:tasks.rados.rados.0.smithi138.stdout:1794: writing smithi138138488-47 from 504441 to 992217 tid 1 2023-06-27T12:45:30.074 INFO:tasks.rados.rados.0.smithi138.stdout:1794: writing smithi138138488-47 from 1467372 to 2203661 tid 2 2023-06-27T12:45:30.074 INFO:tasks.rados.rados.0.smithi138.stdout:1795: write oid 23 current snap is 206 2023-06-27T12:45:30.074 INFO:tasks.rados.rados.0.smithi138.stdout:1795: seq_num 864 ranges {521248=563025,1809501=596300} 2023-06-27T12:45:30.080 INFO:tasks.rados.rados.0.smithi138.stdout:1795: writing smithi138138488-23 from 521248 to 1084273 tid 1 2023-06-27T12:45:30.088 INFO:tasks.rados.rados.0.smithi138.stdout:1795: writing smithi138138488-23 from 1809501 to 2405801 tid 2 2023-06-27T12:45:30.088 INFO:tasks.rados.rados.0.smithi138.stdout:1796: read oid 10 snap 198 2023-06-27T12:45:30.088 INFO:tasks.rados.rados.0.smithi138.stdout:1796: expect (ObjNum 816 snap 198 seq_num 816) 2023-06-27T12:45:30.088 INFO:tasks.rados.rados.0.smithi138.stdout:1797: read oid 6 snap 198 2023-06-27T12:45:30.088 INFO:tasks.rados.rados.0.smithi138.stdout:1797: expect (ObjNum 811 snap 198 seq_num 811) 2023-06-27T12:45:30.089 INFO:tasks.rados.rados.0.smithi138.stdout:1798: snap_create 2023-06-27T12:45:30.089 INFO:tasks.rados.rados.0.smithi138.stdout:1790: expect (ObjNum 769 snap 189 seq_num 769) 2023-06-27T12:45:30.278 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.b] on osd.0 to force-recovery; instructing pg(s) [3.7] on osd.1 to force-recovery; instructing pg(s) [3.0,3.4,3.e] on osd.2 to force-recovery; 2023-06-27T12:45:30.279 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:45:30.294 INFO:tasks.rados.rados.0.smithi138.stdout:1793: finishing write tid 1 to smithi138138488-43 2023-06-27T12:45:30.294 INFO:tasks.rados.rados.0.smithi138.stdout:1792: finishing write tid 1 to smithi138138488-36 2023-06-27T12:45:30.294 INFO:tasks.rados.rados.0.smithi138.stdout:1794: finishing write tid 1 to smithi138138488-47 2023-06-27T12:45:30.294 INFO:tasks.rados.rados.0.smithi138.stdout:1794: finishing write tid 2 to smithi138138488-47 2023-06-27T12:45:30.295 INFO:tasks.rados.rados.0.smithi138.stdout:1794: finishing write tid 3 to smithi138138488-47 2023-06-27T12:45:30.295 INFO:tasks.rados.rados.0.smithi138.stdout:1794: finishing write tid 4 to smithi138138488-47 2023-06-27T12:45:30.295 INFO:tasks.rados.rados.0.smithi138.stdout:1794: oid 47 updating version 0 to 341 2023-06-27T12:45:30.295 INFO:tasks.rados.rados.0.smithi138.stdout:1794: oid 47 version 341 is already newer than 341 2023-06-27T12:45:30.295 INFO:tasks.rados.rados.0.smithi138.stdout:1794: oid 47 version 341 is already newer than 341 2023-06-27T12:45:30.295 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 47 v 341 (ObjNum 863 snap 206 seq_num 863) dirty exists 2023-06-27T12:45:30.295 INFO:tasks.rados.rados.0.smithi138.stdout:1794: left oid 47 (ObjNum 863 snap 206 seq_num 863) 2023-06-27T12:45:30.295 INFO:tasks.rados.rados.0.smithi138.stdout:1792: finishing write tid 2 to smithi138138488-36 2023-06-27T12:45:30.296 INFO:tasks.rados.rados.0.smithi138.stdout:1796: expect (ObjNum 816 snap 198 seq_num 816) 2023-06-27T12:45:30.591 INFO:tasks.rados.rados.0.smithi138.stdout:1793: finishing write tid 2 to smithi138138488-43 2023-06-27T12:45:30.592 INFO:tasks.rados.rados.0.smithi138.stdout:1793: finishing write tid 3 to smithi138138488-43 2023-06-27T12:45:30.592 INFO:tasks.rados.rados.0.smithi138.stdout:1795: finishing write tid 1 to smithi138138488-23 2023-06-27T12:45:30.592 INFO:tasks.rados.rados.0.smithi138.stdout:1793: finishing write tid 4 to smithi138138488-43 2023-06-27T12:45:30.592 INFO:tasks.rados.rados.0.smithi138.stdout:1793: oid 43 updating version 0 to 332 2023-06-27T12:45:30.592 INFO:tasks.rados.rados.0.smithi138.stdout:1793: oid 43 version 332 is already newer than 330 2023-06-27T12:45:30.593 INFO:tasks.rados.rados.0.smithi138.stdout:1793: oid 43 version 332 is already newer than 332 2023-06-27T12:45:30.593 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 43 v 332 (ObjNum 862 snap 206 seq_num 862) dirty exists 2023-06-27T12:45:30.593 INFO:tasks.rados.rados.0.smithi138.stdout:1793: left oid 43 (ObjNum 862 snap 206 seq_num 862) 2023-06-27T12:45:30.593 INFO:tasks.rados.rados.0.smithi138.stdout:1795: finishing write tid 2 to smithi138138488-23 2023-06-27T12:45:30.593 INFO:tasks.rados.rados.0.smithi138.stdout:1790: done (8 left) 2023-06-27T12:45:30.593 INFO:tasks.rados.rados.0.smithi138.stdout:1791: done (7 left) 2023-06-27T12:45:30.593 INFO:tasks.rados.rados.0.smithi138.stdout:1793: done (6 left) 2023-06-27T12:45:30.593 INFO:tasks.rados.rados.0.smithi138.stdout:1794: done (5 left) 2023-06-27T12:45:30.594 INFO:tasks.rados.rados.0.smithi138.stdout:1796: done (4 left) 2023-06-27T12:45:30.594 INFO:tasks.rados.rados.0.smithi138.stdout:1798: done (3 left) 2023-06-27T12:45:30.594 INFO:tasks.rados.rados.0.smithi138.stdout:1799: snap_remove snap 206 2023-06-27T12:45:31.334 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:31.468 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:31.463+0000 7f0e1115e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:31.471 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:31.467+0000 7f0e1115e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:31.515 INFO:tasks.rados.rados.0.smithi138.stdout:1799: done (3 left) 2023-06-27T12:45:31.516 INFO:tasks.rados.rados.0.smithi138.stdout:1800: delete oid 50 current snap is 207 2023-06-27T12:45:31.516 INFO:tasks.rados.rados.0.smithi138.stdout:1795: finishing write tid 3 to smithi138138488-23 2023-06-27T12:45:31.516 INFO:tasks.rados.rados.0.smithi138.stdout:1795: finishing write tid 4 to smithi138138488-23 2023-06-27T12:45:31.516 INFO:tasks.rados.rados.0.smithi138.stdout:1795: oid 23 updating version 0 to 259 2023-06-27T12:45:31.516 INFO:tasks.rados.rados.0.smithi138.stdout:1795: oid 23 version 259 is already newer than 259 2023-06-27T12:45:31.517 INFO:tasks.rados.rados.0.smithi138.stdout:1795: oid 23 version 259 is already newer than 259 2023-06-27T12:45:31.517 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 259 (ObjNum 864 snap 206 seq_num 864) dirty exists 2023-06-27T12:45:31.517 INFO:tasks.rados.rados.0.smithi138.stdout:1795: left oid 23 (ObjNum 864 snap 206 seq_num 864) 2023-06-27T12:45:31.517 INFO:tasks.rados.rados.0.smithi138.stdout:1792: finishing write tid 3 to smithi138138488-36 2023-06-27T12:45:31.518 INFO:tasks.rados.rados.0.smithi138.stdout:1792: finishing write tid 4 to smithi138138488-36 2023-06-27T12:45:31.518 INFO:tasks.rados.rados.0.smithi138.stdout:1792: oid 36 updating version 0 to 408 2023-06-27T12:45:31.518 INFO:tasks.rados.rados.0.smithi138.stdout:1792: oid 36 version 408 is already newer than 408 2023-06-27T12:45:31.518 INFO:tasks.rados.rados.0.smithi138.stdout:1792: oid 36 version 408 is already newer than 407 2023-06-27T12:45:31.518 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 36 v 408 (ObjNum 861 snap 206 seq_num 861) dirty exists 2023-06-27T12:45:31.518 INFO:tasks.rados.rados.0.smithi138.stdout:1792: left oid 36 (ObjNum 861 snap 206 seq_num 861) 2023-06-27T12:45:31.519 INFO:tasks.rados.rados.0.smithi138.stdout:1797: expect (ObjNum 811 snap 198 seq_num 811) 2023-06-27T12:45:31.519 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:31.524 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:31.525 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:31.727 INFO:tasks.rados.rados.0.smithi138.stdout:1792: done (3 left) 2023-06-27T12:45:31.727 INFO:tasks.rados.rados.0.smithi138.stdout:1795: done (2 left) 2023-06-27T12:45:31.728 INFO:tasks.rados.rados.0.smithi138.stdout:1797: done (1 left) 2023-06-27T12:45:31.728 INFO:tasks.rados.rados.0.smithi138.stdout:1800: done (0 left) 2023-06-27T12:45:31.728 INFO:tasks.rados.rados.0.smithi138.stdout:1801: write (excl) oid 31 current snap is 207 2023-06-27T12:45:31.728 INFO:tasks.rados.rados.0.smithi138.stdout:1801: seq_num 866 ranges {554862=713523,1926465=599969} 2023-06-27T12:45:31.734 INFO:tasks.rados.rados.0.smithi138.stdout:1801: writing smithi138138488-31 from 554862 to 1268385 tid 1 2023-06-27T12:45:31.741 INFO:tasks.rados.rados.0.smithi138.stdout:1801: writing smithi138138488-31 from 1926465 to 2526434 tid 2 2023-06-27T12:45:31.742 INFO:tasks.rados.rados.0.smithi138.stdout:1802: write (excl) oid 44 current snap is 207 2023-06-27T12:45:31.742 INFO:tasks.rados.rados.0.smithi138.stdout:1802: seq_num 867 ranges {571669=505125,1701300=743627} 2023-06-27T12:45:31.747 INFO:tasks.rados.rados.0.smithi138.stdout:1802: writing smithi138138488-44 from 571669 to 1076794 tid 1 2023-06-27T12:45:31.755 INFO:tasks.rados.rados.0.smithi138.stdout:1802: writing smithi138138488-44 from 1701300 to 2444927 tid 2 2023-06-27T12:45:31.755 INFO:tasks.rados.rados.0.smithi138.stdout:1803: delete oid 7 current snap is 207 2023-06-27T12:45:31.840 INFO:tasks.rados.rados.0.smithi138.stdout:1801: finishing write tid 1 to smithi138138488-31 2023-06-27T12:45:31.856 INFO:tasks.rados.rados.0.smithi138.stdout:1803: done (2 left) 2023-06-27T12:45:31.856 INFO:tasks.rados.rados.0.smithi138.stdout:1804: snap_create 2023-06-27T12:45:31.858 INFO:tasks.rados.rados.0.smithi138.stdout:1801: finishing write tid 2 to smithi138138488-31 2023-06-27T12:45:31.901 INFO:tasks.rados.rados.0.smithi138.stdout:1801: finishing write tid 3 to smithi138138488-31 2023-06-27T12:45:31.902 INFO:tasks.rados.rados.0.smithi138.stdout:1801: finishing write tid 4 to smithi138138488-31 2023-06-27T12:45:31.902 INFO:tasks.rados.rados.0.smithi138.stdout:1801: oid 31 updating version 0 to 481 2023-06-27T12:45:31.902 INFO:tasks.rados.rados.0.smithi138.stdout:1801: oid 31 version 481 is already newer than 481 2023-06-27T12:45:31.902 INFO:tasks.rados.rados.0.smithi138.stdout:1801: oid 31 version 481 is already newer than 478 2023-06-27T12:45:31.902 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 481 (ObjNum 866 snap 207 seq_num 866) dirty exists 2023-06-27T12:45:31.902 INFO:tasks.rados.rados.0.smithi138.stdout:1801: left oid 31 (ObjNum 866 snap 207 seq_num 866) 2023-06-27T12:45:31.947 INFO:tasks.rados.rados.0.smithi138.stdout:1802: finishing write tid 1 to smithi138138488-44 2023-06-27T12:45:31.998 INFO:tasks.rados.rados.0.smithi138.stdout:1802: finishing write tid 2 to smithi138138488-44 2023-06-27T12:45:31.998 INFO:tasks.rados.rados.0.smithi138.stdout:1802: finishing write tid 3 to smithi138138488-44 2023-06-27T12:45:31.999 INFO:tasks.rados.rados.0.smithi138.stdout:1802: finishing write tid 4 to smithi138138488-44 2023-06-27T12:45:31.999 INFO:tasks.rados.rados.0.smithi138.stdout:1802: oid 44 updating version 0 to 484 2023-06-27T12:45:31.999 INFO:tasks.rados.rados.0.smithi138.stdout:1802: oid 44 version 484 is already newer than 484 2023-06-27T12:45:32.000 INFO:tasks.rados.rados.0.smithi138.stdout:1802: oid 44 version 484 is already newer than 482 2023-06-27T12:45:32.000 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 44 v 484 (ObjNum 867 snap 207 seq_num 867) dirty exists 2023-06-27T12:45:32.000 INFO:tasks.rados.rados.0.smithi138.stdout:1802: left oid 44 (ObjNum 867 snap 207 seq_num 867) 2023-06-27T12:45:32.521 INFO:tasks.rados.rados.0.smithi138.stdout:1801: done (2 left) 2023-06-27T12:45:32.522 INFO:tasks.rados.rados.0.smithi138.stdout:1802: done (1 left) 2023-06-27T12:45:32.522 INFO:tasks.rados.rados.0.smithi138.stdout:1804: done (0 left) 2023-06-27T12:45:32.522 INFO:tasks.rados.rados.0.smithi138.stdout:1805: write (excl) oid 32 current snap is 208 2023-06-27T12:45:32.522 INFO:tasks.rados.rados.0.smithi138.stdout:1805: seq_num 869 ranges {605283=655623,1701911=747296} 2023-06-27T12:45:32.528 INFO:tasks.rados.rados.0.smithi138.stdout:1805: writing smithi138138488-32 from 605283 to 1260906 tid 1 2023-06-27T12:45:32.535 INFO:tasks.rados.rados.0.smithi138.stdout:1805: writing smithi138138488-32 from 1701911 to 2449207 tid 2 2023-06-27T12:45:32.536 INFO:tasks.rados.rados.0.smithi138.stdout:1806: write oid 7 current snap is 208 2023-06-27T12:45:32.536 INFO:tasks.rados.rados.0.smithi138.stdout:1806: seq_num 870 ranges {622090=730872,1760393=607307} 2023-06-27T12:45:32.537 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~297s 2023-06-27T12:45:32.543 INFO:tasks.rados.rados.0.smithi138.stdout:1806: writing smithi138138488-7 from 622090 to 1352962 tid 1 2023-06-27T12:45:32.549 INFO:tasks.rados.rados.0.smithi138.stdout:1806: writing smithi138138488-7 from 1760393 to 2367700 tid 2 2023-06-27T12:45:32.550 INFO:tasks.rados.rados.0.smithi138.stdout:1807: delete oid 27 current snap is 208 2023-06-27T12:45:32.601 INFO:tasks.rados.rados.0.smithi138.stdout:1806: finishing write tid 1 to smithi138138488-7 2023-06-27T12:45:32.650 INFO:tasks.rados.rados.0.smithi138.stdout:1806: finishing write tid 2 to smithi138138488-7 2023-06-27T12:45:32.650 INFO:tasks.rados.rados.0.smithi138.stdout:1806: finishing write tid 3 to smithi138138488-7 2023-06-27T12:45:32.651 INFO:tasks.rados.rados.0.smithi138.stdout:1805: finishing write tid 1 to smithi138138488-32 2023-06-27T12:45:32.652 INFO:tasks.rados.rados.0.smithi138.stdout:1806: finishing write tid 4 to smithi138138488-7 2023-06-27T12:45:32.652 INFO:tasks.rados.rados.0.smithi138.stdout:1806: oid 7 updating version 0 to 338 2023-06-27T12:45:32.652 INFO:tasks.rados.rados.0.smithi138.stdout:1806: oid 7 version 338 is already newer than 338 2023-06-27T12:45:32.652 INFO:tasks.rados.rados.0.smithi138.stdout:1806: oid 7 version 338 is already newer than 338 2023-06-27T12:45:32.652 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 7 v 338 (ObjNum 870 snap 208 seq_num 870) dirty exists 2023-06-27T12:45:32.652 INFO:tasks.rados.rados.0.smithi138.stdout:1806: left oid 7 (ObjNum 870 snap 208 seq_num 870) 2023-06-27T12:45:32.661 INFO:tasks.rados.rados.0.smithi138.stdout:1806: done (2 left) 2023-06-27T12:45:32.662 INFO:tasks.rados.rados.0.smithi138.stdout:1807: done (1 left) 2023-06-27T12:45:32.662 INFO:tasks.rados.rados.0.smithi138.stdout:1808: read oid 30 snap -1 2023-06-27T12:45:32.662 INFO:tasks.rados.rados.0.smithi138.stdout:1808: expect (ObjNum 850 snap 205 seq_num 850) 2023-06-27T12:45:32.662 INFO:tasks.rados.rados.0.smithi138.stdout:1809: read oid 10 snap -1 2023-06-27T12:45:32.662 INFO:tasks.rados.rados.0.smithi138.stdout:1809: expect deleted 2023-06-27T12:45:32.662 INFO:tasks.rados.rados.0.smithi138.stdout:1810: write (excl) oid 31 current snap is 208 2023-06-27T12:45:32.662 INFO:tasks.rados.rados.0.smithi138.stdout:1810: seq_num 872 ranges {655704=481370,1761004=494623} 2023-06-27T12:45:32.667 INFO:tasks.rados.rados.0.smithi138.stdout:1810: writing smithi138138488-31 from 655704 to 1137074 tid 1 2023-06-27T12:45:32.674 INFO:tasks.rados.rados.0.smithi138.stdout:1810: writing smithi138138488-31 from 1761004 to 2255627 tid 2 2023-06-27T12:45:32.675 INFO:tasks.rados.rados.0.smithi138.stdout:1811: write oid 34 current snap is 208 2023-06-27T12:45:32.675 INFO:tasks.rados.rados.0.smithi138.stdout:1811: seq_num 873 ranges {672511=556619,1819486=754634} 2023-06-27T12:45:32.682 INFO:tasks.rados.rados.0.smithi138.stdout:1811: writing smithi138138488-34 from 672511 to 1229130 tid 1 2023-06-27T12:45:32.690 INFO:tasks.rados.rados.0.smithi138.stdout:1811: writing smithi138138488-34 from 1819486 to 2574120 tid 2 2023-06-27T12:45:32.690 INFO:tasks.rados.rados.0.smithi138.stdout:1812: write oid 35 current snap is 208 2023-06-27T12:45:32.691 INFO:tasks.rados.rados.0.smithi138.stdout:1812: seq_num 874 ranges {689318=631868,1877968=498292} 2023-06-27T12:45:32.698 INFO:tasks.rados.rados.0.smithi138.stdout:1812: writing smithi138138488-35 from 689318 to 1321186 tid 1 2023-06-27T12:45:32.704 INFO:tasks.rados.rados.0.smithi138.stdout:1812: writing smithi138138488-35 from 1877968 to 2376260 tid 2 2023-06-27T12:45:32.704 INFO:tasks.rados.rados.0.smithi138.stdout:1809: done (5 left) 2023-06-27T12:45:32.705 INFO:tasks.rados.rados.0.smithi138.stdout:1813: write (excl) oid 19 current snap is 208 2023-06-27T12:45:32.705 INFO:tasks.rados.rados.0.smithi138.stdout:1805: finishing write tid 2 to smithi138138488-32 2023-06-27T12:45:32.705 INFO:tasks.rados.rados.0.smithi138.stdout:1813: seq_num 875 ranges {706125=423470,1652803=758303} 2023-06-27T12:45:32.710 INFO:tasks.rados.rados.0.smithi138.stdout:1813: writing smithi138138488-19 from 706125 to 1129595 tid 1 2023-06-27T12:45:32.718 INFO:tasks.rados.rados.0.smithi138.stdout:1813: writing smithi138138488-19 from 1652803 to 2411106 tid 2 2023-06-27T12:45:32.718 INFO:tasks.rados.rados.0.smithi138.stdout:1814: delete oid 29 current snap is 208 2023-06-27T12:45:32.718 INFO:tasks.rados.rados.0.smithi138.stdout:1805: finishing write tid 3 to smithi138138488-32 2023-06-27T12:45:32.719 INFO:tasks.rados.rados.0.smithi138.stdout:1805: finishing write tid 4 to smithi138138488-32 2023-06-27T12:45:32.719 INFO:tasks.rados.rados.0.smithi138.stdout:1805: oid 32 updating version 0 to 546 2023-06-27T12:45:32.719 INFO:tasks.rados.rados.0.smithi138.stdout:1805: oid 32 updating version 546 to 548 2023-06-27T12:45:32.719 INFO:tasks.rados.rados.0.smithi138.stdout:1805: oid 32 version 548 is already newer than 548 2023-06-27T12:45:32.719 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 32 v 548 (ObjNum 869 snap 208 seq_num 869) dirty exists 2023-06-27T12:45:32.719 INFO:tasks.rados.rados.0.smithi138.stdout:1805: left oid 32 (ObjNum 869 snap 208 seq_num 869) 2023-06-27T12:45:32.725 INFO:tasks.rados.rados.0.smithi138.stdout:1810: finishing write tid 1 to smithi138138488-31 2023-06-27T12:45:32.736 INFO:tasks.rados.rados.0.smithi138.stdout:1808: expect (ObjNum 850 snap 205 seq_num 850) 2023-06-27T12:45:32.865 INFO:tasks.rados.rados.0.smithi138.stdout:1813: finishing write tid 1 to smithi138138488-19 2023-06-27T12:45:32.865 INFO:tasks.rados.rados.0.smithi138.stdout:1812: finishing write tid 1 to smithi138138488-35 2023-06-27T12:45:32.865 INFO:tasks.rados.rados.0.smithi138.stdout:1812: finishing write tid 2 to smithi138138488-35 2023-06-27T12:45:32.865 INFO:tasks.rados.rados.0.smithi138.stdout:1810: finishing write tid 2 to smithi138138488-31 2023-06-27T12:45:32.865 INFO:tasks.rados.rados.0.smithi138.stdout:1805: done (6 left) 2023-06-27T12:45:32.865 INFO:tasks.rados.rados.0.smithi138.stdout:1808: done (5 left) 2023-06-27T12:45:32.866 INFO:tasks.rados.rados.0.smithi138.stdout:1814: done (4 left) 2023-06-27T12:45:32.866 INFO:tasks.rados.rados.0.smithi138.stdout:1815: snap_remove snap 198 2023-06-27T12:45:33.524 INFO:tasks.rados.rados.0.smithi138.stdout:1815: done (4 left) 2023-06-27T12:45:33.525 INFO:tasks.rados.rados.0.smithi138.stdout:1816: snap_create 2023-06-27T12:45:33.525 INFO:tasks.rados.rados.0.smithi138.stdout:1812: finishing write tid 3 to smithi138138488-35 2023-06-27T12:45:33.526 INFO:tasks.rados.rados.0.smithi138.stdout:1812: finishing write tid 4 to smithi138138488-35 2023-06-27T12:45:33.526 INFO:tasks.rados.rados.0.smithi138.stdout:1812: oid 35 updating version 0 to 552 2023-06-27T12:45:33.526 INFO:tasks.rados.rados.0.smithi138.stdout:1812: oid 35 version 552 is already newer than 552 2023-06-27T12:45:33.526 INFO:tasks.rados.rados.0.smithi138.stdout:1812: oid 35 version 552 is already newer than 552 2023-06-27T12:45:33.526 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 552 (ObjNum 874 snap 208 seq_num 874) dirty exists 2023-06-27T12:45:33.527 INFO:tasks.rados.rados.0.smithi138.stdout:1812: left oid 35 (ObjNum 874 snap 208 seq_num 874) 2023-06-27T12:45:33.527 INFO:tasks.rados.rados.0.smithi138.stdout:1813: finishing write tid 2 to smithi138138488-19 2023-06-27T12:45:33.527 INFO:tasks.rados.rados.0.smithi138.stdout:1813: finishing write tid 3 to smithi138138488-19 2023-06-27T12:45:33.527 INFO:tasks.rados.rados.0.smithi138.stdout:1813: finishing write tid 4 to smithi138138488-19 2023-06-27T12:45:33.527 INFO:tasks.rados.rados.0.smithi138.stdout:1813: oid 19 updating version 0 to 566 2023-06-27T12:45:33.528 INFO:tasks.rados.rados.0.smithi138.stdout:1813: oid 19 version 566 is already newer than 563 2023-06-27T12:45:33.528 INFO:tasks.rados.rados.0.smithi138.stdout:1813: oid 19 version 566 is already newer than 566 2023-06-27T12:45:33.528 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 19 v 566 (ObjNum 875 snap 208 seq_num 875) dirty exists 2023-06-27T12:45:33.529 INFO:tasks.rados.rados.0.smithi138.stdout:1813: left oid 19 (ObjNum 875 snap 208 seq_num 875) 2023-06-27T12:45:33.529 INFO:tasks.rados.rados.0.smithi138.stdout:1810: finishing write tid 3 to smithi138138488-31 2023-06-27T12:45:33.529 INFO:tasks.rados.rados.0.smithi138.stdout:1810: finishing write tid 4 to smithi138138488-31 2023-06-27T12:45:33.529 INFO:tasks.rados.rados.0.smithi138.stdout:1810: oid 31 updating version 0 to 485 2023-06-27T12:45:33.529 INFO:tasks.rados.rados.0.smithi138.stdout:1810: oid 31 updating version 485 to 488 2023-06-27T12:45:33.529 INFO:tasks.rados.rados.0.smithi138.stdout:1810: oid 31 version 488 is already newer than 488 2023-06-27T12:45:33.530 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 31 v 488 (ObjNum 872 snap 208 seq_num 872) dirty exists 2023-06-27T12:45:33.530 INFO:tasks.rados.rados.0.smithi138.stdout:1810: left oid 31 (ObjNum 872 snap 208 seq_num 872) 2023-06-27T12:45:33.530 INFO:tasks.rados.rados.0.smithi138.stdout:1811: finishing write tid 1 to smithi138138488-34 2023-06-27T12:45:33.530 INFO:tasks.rados.rados.0.smithi138.stdout:1811: finishing write tid 2 to smithi138138488-34 2023-06-27T12:45:33.530 INFO:tasks.rados.rados.0.smithi138.stdout:1811: finishing write tid 3 to smithi138138488-34 2023-06-27T12:45:33.530 INFO:tasks.rados.rados.0.smithi138.stdout:1811: finishing write tid 4 to smithi138138488-34 2023-06-27T12:45:33.530 INFO:tasks.rados.rados.0.smithi138.stdout:1811: oid 34 updating version 0 to 492 2023-06-27T12:45:33.531 INFO:tasks.rados.rados.0.smithi138.stdout:1811: oid 34 version 492 is already newer than 492 2023-06-27T12:45:33.531 INFO:tasks.rados.rados.0.smithi138.stdout:1811: oid 34 version 492 is already newer than 492 2023-06-27T12:45:33.531 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 34 v 492 (ObjNum 873 snap 208 seq_num 873) dirty exists 2023-06-27T12:45:33.531 INFO:tasks.rados.rados.0.smithi138.stdout:1811: left oid 34 (ObjNum 873 snap 208 seq_num 873) 2023-06-27T12:45:33.531 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:45:33.659 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:33.653+0000 7f46c1c67700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:33.661 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:33.657+0000 7f46c1c67700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:33.708 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:33.709 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:34.525 INFO:tasks.rados.rados.0.smithi138.stdout:1810: done (4 left) 2023-06-27T12:45:34.525 INFO:tasks.rados.rados.0.smithi138.stdout:1811: done (3 left) 2023-06-27T12:45:34.525 INFO:tasks.rados.rados.0.smithi138.stdout:1812: done (2 left) 2023-06-27T12:45:34.525 INFO:tasks.rados.rados.0.smithi138.stdout:1813: done (1 left) 2023-06-27T12:45:34.525 INFO:tasks.rados.rados.0.smithi138.stdout:1816: done (0 left) 2023-06-27T12:45:34.526 INFO:tasks.rados.rados.0.smithi138.stdout:1817: read oid 41 snap -1 2023-06-27T12:45:34.526 INFO:tasks.rados.rados.0.smithi138.stdout:1817: expect deleted 2023-06-27T12:45:34.526 INFO:tasks.rados.rados.0.smithi138.stdout:1818: snap_remove snap 207 2023-06-27T12:45:35.291 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:45:35.291 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:45:35.291 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:45:35.425 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:35.417+0000 7f660ec6e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:35.427 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:35.424+0000 7f660ec6e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:35.479 INFO:tasks.rados.rados.0.smithi138.stdout:1818: done (1 left) 2023-06-27T12:45:35.479 INFO:tasks.rados.rados.0.smithi138.stdout:1819: delete oid 31 current snap is 209 2023-06-27T12:45:35.655 INFO:tasks.rados.rados.0.smithi138.stdout:1817: done (1 left) 2023-06-27T12:45:35.655 INFO:tasks.rados.rados.0.smithi138.stdout:1819: done (0 left) 2023-06-27T12:45:35.655 INFO:tasks.rados.rados.0.smithi138.stdout:1820: snap_create 2023-06-27T12:45:35.660 INFO:teuthology.orchestra.run.smithi138.stdout: 2023-06-27T12:45:35.662 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2023-06-27T12:45:35.673 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":663,"stamp":"2023-06-27T12:45:35.473921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88327022,"num_objects":481,"num_object_clones":434,"num_object_copies":1443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":2767,"num_read_kb":2072006,"num_write":2505,"num_write_kb":732676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4740,"ondisk_log_size":4740,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1365908,"kb_used_data":1216728,"kb_used_omap":0,"kb_used_meta":148224,"kb_avail":376121452,"statfs":{"total":386547056640,"available":385148366848,"internally_reserved":0,"allocated":1245929472,"data_stored":1452245720,"data_compressed":89706,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":151781376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4235034,"num_objects":11,"num_object_clones":11,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":27463,"num_write":56,"num_write_kb":16764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.963637"},"pg_stats":[{"pgid":"3.f","version":"424'83","reported_seq":1062,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.818365+0000","last_change":"2023-06-27T12:45:33.818720+0000","last_active":"2023-06-27T12:45:33.818365+0000","last_peered":"2023-06-27T12:45:33.818365+0000","last_clean":"2023-06-27T12:40:36.586313+0000","last_became_active":"2023-06-27T12:41:00.516051+0000","last_became_peered":"2023-06-27T12:41:00.516051+0000","last_unstale":"2023-06-27T12:45:33.818365+0000","last_undegraded":"2023-06-27T12:41:38.731059+0000","last_fullsized":"2023-06-27T12:41:00.437887+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2050652,"num_objects":11,"num_object_clones":8,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":28,"num_read_kb":6254,"num_write":51,"num_write_kb":14737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":11}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"437'343","reported_seq":1388,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.549445+0000","last_change":"2023-06-27T12:45:33.550330+0000","last_active":"2023-06-27T12:45:33.549445+0000","last_peered":"2023-06-27T12:45:33.549445+0000","last_clean":"2023-06-27T12:40:36.443381+0000","last_became_active":"2023-06-27T12:41:00.484240+0000","last_became_peered":"2023-06-27T12:41:00.484240+0000","last_unstale":"2023-06-27T12:45:33.549445+0000","last_undegraded":"2023-06-27T12:41:00.416704+0000","last_fullsized":"2023-06-27T12:41:00.415835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7681360,"num_objects":40,"num_object_clones":36,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":189,"num_read_kb":223844,"num_write":210,"num_write_kb":63014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":40}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"434'408","reported_seq":1404,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.971192+0000","last_change":"2023-06-27T12:45:33.971720+0000","last_active":"2023-06-27T12:45:33.971192+0000","last_peered":"2023-06-27T12:45:33.971192+0000","last_clean":"2023-06-27T12:40:36.578923+0000","last_became_active":"2023-06-27T12:41:00.507815+0000","last_became_peered":"2023-06-27T12:41:00.507815+0000","last_unstale":"2023-06-27T12:45:33.971192+0000","last_undegraded":"2023-06-27T12:41:00.418434+0000","last_fullsized":"2023-06-27T12:41:00.417656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4874426,"num_objects":45,"num_object_clones":40,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":328,"num_read_kb":224753,"num_write":243,"num_write_kb":71822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":45}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"402'248","reported_seq":1293,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.818273+0000","last_change":"2023-06-27T12:45:33.830064+0000","last_active":"2023-06-27T12:45:33.818273+0000","last_peered":"2023-06-27T12:45:33.818273+0000","last_clean":"2023-06-27T12:40:36.592661+0000","last_became_active":"2023-06-27T12:41:00.551554+0000","last_became_peered":"2023-06-27T12:41:00.551554+0000","last_unstale":"2023-06-27T12:45:33.818273+0000","last_undegraded":"2023-06-27T12:41:00.444507+0000","last_fullsized":"2023-06-27T12:41:00.443294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8874591,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":246,"num_read_kb":188083,"num_write":156,"num_write_kb":44779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"427'134","reported_seq":803,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.822108+0000","last_change":"2023-06-27T12:45:33.822490+0000","last_active":"2023-06-27T12:45:33.822108+0000","last_peered":"2023-06-27T12:45:33.822108+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.525624+0000","last_became_peered":"2023-06-27T12:41:00.525624+0000","last_unstale":"2023-06-27T12:45:33.822108+0000","last_undegraded":"2023-06-27T12:41:35.036421+0000","last_fullsized":"2023-06-27T12:41:00.438132+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":12,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":143,"num_read_kb":96068,"num_write":68,"num_write_kb":20943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":13}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"437'492","reported_seq":1389,"reported_epoch":438,"state":"active+clean","last_fresh":"2023-06-27T12:45:33.842055+0000","last_change":"2023-06-27T12:45:33.842180+0000","last_active":"2023-06-27T12:45:33.842055+0000","last_peered":"2023-06-27T12:45:33.842055+0000","last_clean":"2023-06-27T12:45:33.842055+0000","last_became_active":"2023-06-27T12:38:04.532783+0000","last_became_peered":"2023-06-27T12:38:04.532783+0000","last_unstale":"2023-06-27T12:45:33.842055+0000","last_undegraded":"2023-06-27T12:45:33.842055+0000","last_fullsized":"2023-06-27T12:45:33.842055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12354102,"num_objects":54,"num_object_clones":48,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":380,"num_read_kb":241469,"num_write":294,"num_write_kb":84115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"437'338","reported_seq":1216,"reported_epoch":438,"state":"active+clean","last_fresh":"2023-06-27T12:45:33.882628+0000","last_change":"2023-06-27T12:45:33.882749+0000","last_active":"2023-06-27T12:45:33.882628+0000","last_peered":"2023-06-27T12:45:33.882628+0000","last_clean":"2023-06-27T12:45:33.882628+0000","last_became_active":"2023-06-27T12:38:04.543999+0000","last_became_peered":"2023-06-27T12:38:04.543999+0000","last_unstale":"2023-06-27T12:45:33.882628+0000","last_undegraded":"2023-06-27T12:45:33.882628+0000","last_fullsized":"2023-06-27T12:45:33.882628+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6403830,"num_objects":38,"num_object_clones":35,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":186,"num_read_kb":124296,"num_write":186,"num_write_kb":55186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"422'101","reported_seq":1095,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.961509+0000","last_change":"2023-06-27T12:45:33.961867+0000","last_active":"2023-06-27T12:45:33.961509+0000","last_peered":"2023-06-27T12:45:33.961509+0000","last_clean":"2023-06-27T12:40:36.583954+0000","last_became_active":"2023-06-27T12:41:00.518353+0000","last_became_peered":"2023-06-27T12:41:00.518353+0000","last_unstale":"2023-06-27T12:45:33.961509+0000","last_undegraded":"2023-06-27T12:41:00.430165+0000","last_fullsized":"2023-06-27T12:41:00.428104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":11,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":69,"num_read_kb":47130,"num_write":55,"num_write_kb":16677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":738,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-06-27T12:45:33.530820+0000","last_change":"2023-06-27T12:41:00.507638+0000","last_active":"2023-06-27T12:45:33.530820+0000","last_peered":"2023-06-27T12:45:33.530820+0000","last_clean":"2023-06-27T12:40:36.438579+0000","last_became_active":"2023-06-27T12:41:00.507638+0000","last_became_peered":"2023-06-27T12:41:00.507638+0000","last_unstale":"2023-06-27T12:45:33.530820+0000","last_undegraded":"2023-06-27T12:45:33.530820+0000","last_fullsized":"2023-06-27T12:41:00.425532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":984,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-06-27T12:45:33.959034+0000","last_change":"2023-06-27T12:45:33.969417+0000","last_active":"2023-06-27T12:45:33.959034+0000","last_peered":"2023-06-27T12:45:33.959034+0000","last_clean":"2023-06-27T12:40:36.582638+0000","last_became_active":"2023-06-27T12:41:00.543559+0000","last_became_peered":"2023-06-27T12:41:00.543559+0000","last_unstale":"2023-06-27T12:45:33.959034+0000","last_undegraded":"2023-06-27T12:45:33.959034+0000","last_fullsized":"2023-06-27T12:41:00.449823+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":717,"reported_epoch":438,"state":"active+clean","last_fresh":"2023-06-27T12:45:33.813803+0000","last_change":"2023-06-27T12:37:57.942064+0000","last_active":"2023-06-27T12:45:33.813803+0000","last_peered":"2023-06-27T12:45:33.813803+0000","last_clean":"2023-06-27T12:45:33.813803+0000","last_became_active":"2023-06-27T12:37:57.939235+0000","last_became_peered":"2023-06-27T12:37:57.939235+0000","last_unstale":"2023-06-27T12:45:33.813803+0000","last_undegraded":"2023-06-27T12:45:33.813803+0000","last_fullsized":"2023-06-27T12:45:33.813803+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"434'259","reported_seq":1262,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.553556+0000","last_change":"2023-06-27T12:45:33.554283+0000","last_active":"2023-06-27T12:45:33.553556+0000","last_peered":"2023-06-27T12:45:33.553556+0000","last_clean":"2023-06-27T12:40:36.444317+0000","last_became_active":"2023-06-27T12:41:00.514660+0000","last_became_peered":"2023-06-27T12:41:00.514660+0000","last_unstale":"2023-06-27T12:45:33.553556+0000","last_undegraded":"2023-06-27T12:41:00.431985+0000","last_fullsized":"2023-06-27T12:41:00.431191+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7294799,"num_objects":28,"num_object_clones":23,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":108,"num_read_kb":97987,"num_write":158,"num_write_kb":43974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":712,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-06-27T12:45:33.953278+0000","last_change":"2023-06-27T12:41:00.526200+0000","last_active":"2023-06-27T12:45:33.953278+0000","last_peered":"2023-06-27T12:45:33.953278+0000","last_clean":"2023-06-27T12:40:36.587458+0000","last_became_active":"2023-06-27T12:41:00.526200+0000","last_became_peered":"2023-06-27T12:41:00.526200+0000","last_unstale":"2023-06-27T12:45:33.953278+0000","last_undegraded":"2023-06-27T12:45:33.953278+0000","last_fullsized":"2023-06-27T12:41:00.444891+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"437'552","reported_seq":967,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.530798+0000","last_change":"2023-06-27T12:45:33.601895+0000","last_active":"2023-06-27T12:45:33.530798+0000","last_peered":"2023-06-27T12:45:33.530798+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.502535+0000","last_became_peered":"2023-06-27T12:41:00.502535+0000","last_unstale":"2023-06-27T12:45:33.530798+0000","last_undegraded":"2023-06-27T12:41:00.508673+0000","last_fullsized":"2023-06-27T12:41:00.434751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4773617,"num_objects":35,"num_object_clones":33,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":108,"num_read_kb":92940,"num_write":180,"num_write_kb":55504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":35}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":709,"reported_epoch":438,"state":"active+clean","last_fresh":"2023-06-27T12:45:33.952793+0000","last_change":"2023-06-27T12:37:57.911496+0000","last_active":"2023-06-27T12:45:33.952793+0000","last_peered":"2023-06-27T12:45:33.952793+0000","last_clean":"2023-06-27T12:45:33.952793+0000","last_became_active":"2023-06-27T12:37:57.908676+0000","last_became_peered":"2023-06-27T12:37:57.908676+0000","last_unstale":"2023-06-27T12:45:33.952793+0000","last_undegraded":"2023-06-27T12:45:33.952793+0000","last_fullsized":"2023-06-27T12:45:33.952793+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":469,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-06-27T12:45:33.822918+0000","last_change":"2023-06-27T12:41:00.534485+0000","last_active":"2023-06-27T12:45:33.822918+0000","last_peered":"2023-06-27T12:45:33.822918+0000","last_clean":"2023-06-27T12:37:54.794168+0000","last_became_active":"2023-06-27T12:41:00.534485+0000","last_became_peered":"2023-06-27T12:41:00.534485+0000","last_unstale":"2023-06-27T12:45:33.822918+0000","last_undegraded":"2023-06-27T12:45:33.822918+0000","last_fullsized":"2023-06-27T12:41:00.433166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:54.794168+0000","last_clean_scrub_stamp":"2023-06-27T12:37:54.794168+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"437'566","reported_seq":1537,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.985762+0000","last_change":"2023-06-27T12:45:33.986117+0000","last_active":"2023-06-27T12:45:33.985762+0000","last_peered":"2023-06-27T12:45:33.985762+0000","last_clean":"2023-06-27T12:40:36.579059+0000","last_became_active":"2023-06-27T12:41:00.520671+0000","last_became_peered":"2023-06-27T12:41:00.520671+0000","last_unstale":"2023-06-27T12:45:33.985762+0000","last_undegraded":"2023-06-27T12:41:00.426876+0000","last_fullsized":"2023-06-27T12:41:00.425531+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14505365,"num_objects":65,"num_object_clones":59,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":65,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":381,"num_read_kb":232115,"num_write":359,"num_write_kb":105247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":65}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":717,"reported_epoch":438,"state":"active+clean","last_fresh":"2023-06-27T12:45:33.814279+0000","last_change":"2023-06-27T12:37:57.953049+0000","last_active":"2023-06-27T12:45:33.814279+0000","last_peered":"2023-06-27T12:45:33.814279+0000","last_clean":"2023-06-27T12:45:33.814279+0000","last_became_active":"2023-06-27T12:37:57.948524+0000","last_became_peered":"2023-06-27T12:37:57.948524+0000","last_unstale":"2023-06-27T12:45:33.814279+0000","last_undegraded":"2023-06-27T12:45:33.814279+0000","last_fullsized":"2023-06-27T12:45:33.814279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"432'319","reported_seq":888,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.980521+0000","last_change":"2023-06-27T12:45:33.981341+0000","last_active":"2023-06-27T12:45:33.980521+0000","last_peered":"2023-06-27T12:45:33.980521+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.539394+0000","last_became_peered":"2023-06-27T12:41:00.539394+0000","last_unstale":"2023-06-27T12:45:33.980521+0000","last_undegraded":"2023-06-27T12:41:00.676665+0000","last_fullsized":"2023-06-27T12:41:00.427651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5597160,"num_objects":23,"num_object_clones":23,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":194,"num_read_kb":132374,"num_write":130,"num_write_kb":36466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":23}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":481,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-06-27T12:45:33.528641+0000","last_change":"2023-06-27T12:41:00.473566+0000","last_active":"2023-06-27T12:45:33.528641+0000","last_peered":"2023-06-27T12:45:33.528641+0000","last_clean":"2023-06-27T12:37:56.803555+0000","last_became_active":"2023-06-27T12:41:00.473566+0000","last_became_peered":"2023-06-27T12:41:00.473566+0000","last_unstale":"2023-06-27T12:45:33.528641+0000","last_undegraded":"2023-06-27T12:45:33.528641+0000","last_fullsized":"2023-06-27T12:41:00.415746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"434'332","reported_seq":1338,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.541221+0000","last_change":"2023-06-27T12:45:33.541660+0000","last_active":"2023-06-27T12:45:33.541221+0000","last_peered":"2023-06-27T12:45:33.541221+0000","last_clean":"2023-06-27T12:40:36.449119+0000","last_became_active":"2023-06-27T12:41:00.490912+0000","last_became_peered":"2023-06-27T12:41:00.490912+0000","last_unstale":"2023-06-27T12:45:33.541221+0000","last_undegraded":"2023-06-27T12:41:00.419633+0000","last_fullsized":"2023-06-27T12:41:00.418676+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8370447,"num_objects":39,"num_object_clones":35,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":153,"num_read_kb":160343,"num_write":200,"num_write_kb":57694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":39}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"437'199","reported_seq":810,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.964215+0000","last_change":"2023-06-27T12:45:33.964695+0000","last_active":"2023-06-27T12:45:33.964215+0000","last_peered":"2023-06-27T12:45:33.964215+0000","last_clean":"2023-06-27T12:38:03.365505+0000","last_became_active":"2023-06-27T12:41:00.501311+0000","last_became_peered":"2023-06-27T12:41:00.501311+0000","last_unstale":"2023-06-27T12:45:33.964215+0000","last_undegraded":"2023-06-27T12:41:03.514189+0000","last_fullsized":"2023-06-27T12:41:00.414818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":15,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":68416,"num_write":76,"num_write_kb":21766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":15}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":738,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-06-27T12:45:33.535162+0000","last_change":"2023-06-27T12:41:00.521323+0000","last_active":"2023-06-27T12:45:33.535162+0000","last_peered":"2023-06-27T12:45:33.535162+0000","last_clean":"2023-06-27T12:40:36.450227+0000","last_became_active":"2023-06-27T12:41:00.521323+0000","last_became_peered":"2023-06-27T12:41:00.521323+0000","last_unstale":"2023-06-27T12:45:33.535162+0000","last_undegraded":"2023-06-27T12:45:33.535162+0000","last_fullsized":"2023-06-27T12:41:00.431824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":709,"reported_epoch":438,"state":"active+clean","last_fresh":"2023-06-27T12:45:33.958628+0000","last_change":"2023-06-27T12:37:57.916783+0000","last_active":"2023-06-27T12:45:33.958628+0000","last_peered":"2023-06-27T12:45:33.958628+0000","last_clean":"2023-06-27T12:45:33.958628+0000","last_became_active":"2023-06-27T12:37:57.912499+0000","last_became_peered":"2023-06-27T12:37:57.912499+0000","last_unstale":"2023-06-27T12:45:33.958628+0000","last_undegraded":"2023-06-27T12:45:33.958628+0000","last_fullsized":"2023-06-27T12:45:33.958628+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:37:56.803555+0000","last_clean_scrub_stamp":"2023-06-27T12:37:56.803555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"432'269","reported_seq":1269,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-06-27T12:45:33.539429+0000","last_change":"2023-06-27T12:45:33.539862+0000","last_active":"2023-06-27T12:45:33.539429+0000","last_peered":"2023-06-27T12:45:33.539429+0000","last_clean":"2023-06-27T12:40:36.437010+0000","last_became_active":"2023-06-27T12:41:00.489691+0000","last_became_peered":"2023-06-27T12:41:00.489691+0000","last_unstale":"2023-06-27T12:45:33.539429+0000","last_undegraded":"2023-06-27T12:41:00.421847+0000","last_fullsized":"2023-06-27T12:41:00.420883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-27T12:38:03.365505+0000","last_clean_scrub_stamp":"2023-06-27T12:38:03.365505+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5546673,"num_objects":29,"num_object_clones":26,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":130,"num_read_kb":135934,"num_write":139,"num_write_kb":40752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":29}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":88327022,"num_objects":481,"num_object_clones":434,"num_object_copies":1443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":23,"num_read":2767,"num_read_kb":2072006,"num_write":2505,"num_write_kb":732676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4643,"ondisk_log_size":4643,"up":34,"acting":34,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673115,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181280,"kb_used_data":150360,"kb_used_omap":0,"kb_used_meta":30912,"kb_avail":94190560,"statfs":{"total":96636764160,"available":96451133440,"internally_reserved":0,"allocated":153968640,"data_stored":177039552,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":31653888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":10,"seq":42949673420,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":372536,"kb_used_data":333424,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93999304,"statfs":{"total":96636764160,"available":96255287296,"internally_reserved":0,"allocated":341426176,"data_stored":402454981,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706125,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":394904,"kb_used_data":354868,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93976936,"statfs":{"total":96636764160,"available":96232382464,"internally_reserved":0,"allocated":363384832,"data_stored":422941221,"data_compressed":29902,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":10,"seq":42949673420,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":417188,"kb_used_data":378076,"kb_used_omap":0,"kb_used_meta":39104,"kb_avail":93954652,"statfs":{"total":96636764160,"available":96209563648,"internally_reserved":0,"allocated":387149824,"data_stored":449809966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40042496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-06-27T12:45:35.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.e 3.8 3.2 3.1 3.4 3.7 2023-06-27T12:45:35.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:35.799+0000 7ffb73406700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:35.818 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:35.814+0000 7ffb73406700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:35.848 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:35.843+0000 7f2e9096a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:35.851 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:35.847+0000 7f2e9096a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:35.909 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:35.911 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:35.912 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:instructing pg(s) [3.1,3.2,3.7,3.8] on osd.1 to force-recovery; instructing pg(s) [3.4,3.e] on osd.2 to force-recovery; 2023-06-27T12:45:36.033 INFO:teuthology.orchestra.run.smithi138.stderr: 2023-06-27T12:45:36.532 INFO:tasks.rados.rados.0.smithi138.stdout:1820: done (0 left) 2023-06-27T12:45:36.533 INFO:tasks.rados.rados.0.smithi138.stdout:1821: delete oid 2 current snap is 210 2023-06-27T12:45:36.573 INFO:tasks.rados.rados.0.smithi138.stdout:1821: done (0 left) 2023-06-27T12:45:36.573 INFO:tasks.rados.rados.0.smithi138.stdout:1822: snap_remove snap 208 2023-06-27T12:45:37.538 INFO:tasks.rados.rados.0.smithi138.stdout:1822: done (0 left) 2023-06-27T12:45:37.538 INFO:tasks.rados.rados.0.smithi138.stdout:1823: read oid 49 snap -1 2023-06-27T12:45:37.538 INFO:tasks.rados.rados.0.smithi138.stdout:1823: expect (ObjNum 786 snap 192 seq_num 786) 2023-06-27T12:45:37.538 INFO:tasks.rados.rados.0.smithi138.stdout:1824: read oid 46 snap -1 2023-06-27T12:45:37.538 INFO:tasks.rados.rados.0.smithi138.stdout:1824: expect (ObjNum 758 snap 187 seq_num 758) 2023-06-27T12:45:37.538 INFO:tasks.rados.rados.0.smithi138.stdout:1825: write oid 35 current snap is 210 2023-06-27T12:45:37.539 INFO:tasks.rados.rados.0.smithi138.stdout:1825: seq_num 879 ranges {773353=724466,2170378=602975} 2023-06-27T12:45:37.549 INFO:tasks.rados.rados.0.smithi138.stdout:1825: writing smithi138138488-35 from 773353 to 1497819 tid 1 2023-06-27T12:45:37.559 INFO:tasks.rados.rados.0.smithi138.stdout:1825: writing smithi138138488-35 from 2170378 to 2773353 tid 2 2023-06-27T12:45:37.559 INFO:tasks.rados.rados.0.smithi138.stdout:1826: write oid 23 current snap is 210 2023-06-27T12:45:37.559 INFO:tasks.rados.rados.0.smithi138.stdout:1826: seq_num 880 ranges {790160=799715,2112507=509299} 2023-06-27T12:45:37.570 INFO:tasks.rados.rados.0.smithi138.stdout:1826: writing smithi138138488-23 from 790160 to 1589875 tid 1 2023-06-27T12:45:37.578 INFO:tasks.rados.rados.0.smithi138.stdout:1826: writing smithi138138488-23 from 2112507 to 2621806 tid 2 2023-06-27T12:45:37.579 INFO:tasks.rados.rados.0.smithi138.stdout:1827: delete oid 3 current snap is 210 2023-06-27T12:45:37.614 INFO:tasks.rados.rados.0.smithi138.stdout:1824: expect (ObjNum 758 snap 187 seq_num 758) 2023-06-27T12:45:37.755 INFO:tasks.rados.rados.0.smithi138.stdout:1823: expect (ObjNum 786 snap 192 seq_num 786) 2023-06-27T12:45:37.875 INFO:tasks.rados.rados.0.smithi138.stdout:1826: finishing write tid 1 to smithi138138488-23 2023-06-27T12:45:37.875 INFO:tasks.rados.rados.0.smithi138.stdout:1826: finishing write tid 2 to smithi138138488-23 2023-06-27T12:45:37.876 INFO:tasks.rados.rados.0.smithi138.stdout:1825: finishing write tid 1 to smithi138138488-35 2023-06-27T12:45:37.876 INFO:tasks.rados.rados.0.smithi138.stdout:1825: finishing write tid 2 to smithi138138488-35 2023-06-27T12:45:37.876 INFO:tasks.rados.rados.0.smithi138.stdout:1825: finishing write tid 3 to smithi138138488-35 2023-06-27T12:45:37.876 INFO:tasks.rados.rados.0.smithi138.stdout:1825: finishing write tid 4 to smithi138138488-35 2023-06-27T12:45:37.876 INFO:tasks.rados.rados.0.smithi138.stdout:1825: oid 35 updating version 0 to 563 2023-06-27T12:45:37.876 INFO:tasks.rados.rados.0.smithi138.stdout:1825: oid 35 version 563 is already newer than 563 2023-06-27T12:45:37.876 INFO:tasks.rados.rados.0.smithi138.stdout:1825: oid 35 version 563 is already newer than 563 2023-06-27T12:45:37.877 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 35 v 563 (ObjNum 879 snap 210 seq_num 879) dirty exists 2023-06-27T12:45:37.877 INFO:tasks.rados.rados.0.smithi138.stdout:1825: left oid 35 (ObjNum 879 snap 210 seq_num 879) 2023-06-27T12:45:37.877 INFO:tasks.rados.rados.0.smithi138.stdout:1826: finishing write tid 3 to smithi138138488-23 2023-06-27T12:45:37.877 INFO:tasks.rados.rados.0.smithi138.stdout:1823: done (4 left) 2023-06-27T12:45:37.877 INFO:tasks.rados.rados.0.smithi138.stdout:1824: done (3 left) 2023-06-27T12:45:37.877 INFO:tasks.rados.rados.0.smithi138.stdout:1825: done (2 left) 2023-06-27T12:45:37.877 INFO:tasks.rados.rados.0.smithi138.stdout:1827: done (1 left) 2023-06-27T12:45:37.878 INFO:tasks.rados.rados.0.smithi138.stdout:1828: snap_create 2023-06-27T12:45:37.878 INFO:tasks.rados.rados.0.smithi138.stdout:1826: finishing write tid 4 to smithi138138488-23 2023-06-27T12:45:37.878 INFO:tasks.rados.rados.0.smithi138.stdout:1826: oid 23 updating version 0 to 263 2023-06-27T12:45:37.878 INFO:tasks.rados.rados.0.smithi138.stdout:1826: oid 23 version 263 is already newer than 263 2023-06-27T12:45:37.878 INFO:tasks.rados.rados.0.smithi138.stdout:1826: oid 23 version 263 is already newer than 263 2023-06-27T12:45:37.878 INFO:tasks.rados.rados.0.smithi138.stdout:update_object_version oid 23 v 263 (ObjNum 880 snap 210 seq_num 880) dirty exists 2023-06-27T12:45:37.878 INFO:tasks.rados.rados.0.smithi138.stdout:1826: left oid 23 (ObjNum 880 snap 210 seq_num 880) 2023-06-27T12:45:37.913 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:45:38.038 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:38.033+0000 7f2facd18700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:38.041 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:38.037+0000 7f2facd18700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:38.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~303s 2023-06-27T12:45:38.042 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-06-27T12:45:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.3 2023-06-27T12:45:38.089 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-06-27T12:45:38.089 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:45:38.540 INFO:tasks.rados.rados.0.smithi138.stdout:1826: done (1 left) 2023-06-27T12:45:38.540 INFO:tasks.rados.rados.0.smithi138.stdout:1828: done (0 left) 2023-06-27T12:45:38.540 INFO:tasks.rados.rados.0.smithi138.stdout:1829: read oid 33 snap -1 2023-06-27T12:45:38.540 INFO:tasks.rados.rados.0.smithi138.stdout:1829: expect (ObjNum 766 snap 189 seq_num 766) 2023-06-27T12:45:38.541 INFO:tasks.rados.rados.0.smithi138.stdout:1830: read oid 13 snap -1 2023-06-27T12:45:38.541 INFO:tasks.rados.rados.0.smithi138.stdout:1830: expect deleted 2023-06-27T12:45:38.541 INFO:tasks.rados.rados.0.smithi138.stdout:1831: delete oid 10 current snap is 211 2023-06-27T12:45:38.545 INFO:tasks.ceph.osd.0:Sent signal 15 2023-06-27T12:45:38.545 INFO:tasks.ceph.osd.1:Sent signal 15 2023-06-27T12:45:38.546 INFO:tasks.ceph.osd.2:Sent signal 15 2023-06-27T12:45:38.546 INFO:tasks.ceph.mon.a:Sent signal 15 2023-06-27T12:45:38.546 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-06-27T12:45:38.547 INFO:tasks.ceph.mon.a.smithi138.stderr:2023-06-27T12:45:38.543+0000 7f3f388b4700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 136531) UID: 0 2023-06-27T12:45:38.547 INFO:tasks.ceph.mon.a.smithi138.stderr:2023-06-27T12:45:38.543+0000 7f3f388b4700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-06-27T12:45:38.548 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:45:38.543+0000 7f44955c3700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 136039) UID: 0 2023-06-27T12:45:38.548 INFO:tasks.ceph.mgr.x.smithi159.stderr:2023-06-27T12:45:38.543+0000 7f44955c3700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-06-27T12:45:38.593 INFO:tasks.rados.rados.0.smithi138.stdout:1829: expect (ObjNum 766 snap 189 seq_num 766) 2023-06-27T12:45:40.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:45:40.229 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:40.224+0000 7ff9338d0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:45:41.045 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-06-27T12:45:41.045 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-06-27T12:45:41.046 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-27T12:45:41.181 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:45:41.174+0000 7faea712d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:47:40.128 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T12:47:40.129 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:47:41.081 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T12:47:41.082 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 190, in wrapper return func(self) File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 1401, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 778, in force_cancel_recovery self.force_recovery() File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 747, in force_recovery j = self.ceph_manager.get_pgids_to_force(backfill) File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 2371, in get_pgids_to_force j = self.get_pg_stats(); File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi138 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-06-27T12:47:41.082 ERROR:tasks.thrashosds.thrasher:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 1269, in do_thrash self._do_thrash() File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 190, in wrapper return func(self) File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 1401, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 778, in force_cancel_recovery self.force_recovery() File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 747, in force_recovery j = self.ceph_manager.get_pgids_to_force(backfill) File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 2371, in get_pgids_to_force j = self.get_pg_stats(); File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_fd62a8dfcb345fabed3eea6b3a461f1ca848a239/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi138 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-06-27T12:47:42.130 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:47:42.258 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:47:42.252+0000 7f96a7f52700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:49:42.162 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T12:49:42.163 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:49:44.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:49:44.291 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:49:44.286+0000 7f3af1252700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:51:44.196 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T12:51:44.197 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:51:46.199 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:51:46.326 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:51:46.321+0000 7f7eb0232700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:53:46.231 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T12:53:46.232 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:53:48.233 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:53:48.360 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:53:48.354+0000 7fe414d39700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:55:48.264 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T12:55:48.265 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:55:50.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:55:50.393 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:55:50.388+0000 7f175072c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:57:50.299 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T12:57:50.300 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:57:52.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T12:57:52.430 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:57:52.424+0000 7fa5a8990700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T12:59:52.335 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T12:59:52.336 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T12:59:54.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T12:59:54.464 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T12:59:54.458+0000 7f26c7595700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:01:54.370 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:01:54.370 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:01:56.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:01:56.499 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:01:56.494+0000 7f8887fec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:03:56.405 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:03:56.407 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:03:58.408 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:03:58.535 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:03:58.529+0000 7f523329c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:05:58.440 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:05:58.441 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:06:00.442 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:06:00.570 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:06:00.563+0000 7f5203738700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:08:00.474 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:08:00.475 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:08:02.476 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:08:02.602 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:08:02.596+0000 7fa2f96bd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:10:02.507 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:10:02.509 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:10:04.511 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:10:04.637 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:10:04.632+0000 7f0f33c56700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:12:04.542 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:12:04.543 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:12:06.545 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:12:06.673 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:12:06.667+0000 7f2d916b8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:14:06.577 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:14:06.578 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:14:08.579 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:14:08.706 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:14:08.700+0000 7f5e50c4c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:16:08.611 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:16:08.612 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:16:10.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:16:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:16:10.735+0000 7fd19b653700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:18:10.645 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:18:10.645 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:18:12.647 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:18:12.776 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:18:12.771+0000 7f475b5d9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:20:12.682 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:20:12.683 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:20:14.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:20:14.812 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:20:14.807+0000 7fd416115700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:22:14.717 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:22:14.718 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:22:16.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:22:16.846 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:22:16.841+0000 7f5c40361700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:24:16.750 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:24:16.751 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:24:18.752 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:24:18.879 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:24:18.874+0000 7fb3097e9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:26:18.784 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:26:18.785 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:26:20.786 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:26:20.916 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:26:20.909+0000 7fd829c77700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:28:20.818 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:28:20.820 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:28:22.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:28:22.948 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:28:22.943+0000 7fbdeffd4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:30:22.852 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:30:22.853 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:30:24.855 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:30:24.982 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:30:24.977+0000 7f45412d2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:32:24.887 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:32:24.888 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:32:26.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:32:27.015 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:32:27.010+0000 7fd011a3b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:34:26.921 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:34:26.922 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:34:28.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:34:29.050 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:34:29.045+0000 7f79df373700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:36:28.956 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:36:28.957 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:36:30.958 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:36:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:36:31.078+0000 7f04ece77700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:38:30.990 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:38:30.990 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:38:32.991 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:38:33.118 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:38:33.113+0000 7f2ce24f7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:40:33.023 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:40:33.024 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:40:35.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:40:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:40:35.147+0000 7f96035b9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:42:35.057 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:42:35.058 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:42:37.059 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:42:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:42:37.181+0000 7ff6699b3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:44:37.090 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:44:37.091 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:44:39.092 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:44:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:44:39.214+0000 7f23c6026700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:46:39.124 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:46:39.125 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:46:41.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:46:41.253 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:46:41.249+0000 7f353fc7a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:48:41.158 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:48:41.160 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:48:43.161 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:48:43.287 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:48:43.284+0000 7f2f1a3bf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:50:43.192 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:50:43.193 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:50:45.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:50:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:50:45.317+0000 7f6b5d86b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:52:45.225 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:52:45.226 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:52:47.227 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:52:47.353 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:52:47.350+0000 7f6bbae28700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:54:47.258 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:54:47.259 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:54:49.261 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:54:49.388 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:54:49.385+0000 7f551353d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:56:49.292 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:56:49.293 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:56:51.294 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T13:56:51.421 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:56:51.418+0000 7f7d7e041700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T13:58:51.327 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T13:58:51.328 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T13:58:53.329 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T13:58:53.456 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T13:58:53.454+0000 7fc72717d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:00:53.368 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:00:53.369 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:00:55.370 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:00:55.496 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:00:55.494+0000 7f328f908700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:02:55.401 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:02:55.402 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:02:57.403 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:02:57.528 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:02:57.526+0000 7f11118a6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:04:57.435 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:04:57.436 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:04:59.438 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:04:59.564 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:04:59.562+0000 7f959408a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:06:59.469 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:06:59.470 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:07:01.472 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:07:01.598 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:07:01.597+0000 7f961fed9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:09:01.504 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:09:01.505 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:09:03.506 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:09:03.633 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:09:03.632+0000 7fa1d865a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:11:03.538 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:11:03.539 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:11:05.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:11:05.668 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:11:05.666+0000 7fe338a16700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:13:05.573 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:13:05.574 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:13:07.575 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:13:07.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:13:07.700+0000 7f8ac9720700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:15:07.607 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:15:07.608 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:15:09.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:15:09.736 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:15:09.736+0000 7f7e8e183700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:17:09.641 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:17:09.642 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:17:11.642 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:17:11.769 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:17:11.767+0000 7f9a79f1e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:19:11.674 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:19:11.675 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:19:13.676 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:19:13.803 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:19:13.802+0000 7f7fe592e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:21:13.709 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:21:13.710 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:21:15.711 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:21:15.838 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:21:15.837+0000 7f7599738700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:23:15.743 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:23:15.744 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:23:17.745 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:23:17.871 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:23:17.870+0000 7ff6317c4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:25:17.776 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:25:17.777 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:25:19.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:25:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:25:19.905+0000 7f26c4df8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:27:19.811 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:27:19.812 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:27:21.813 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:27:21.940 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:27:21.939+0000 7f5dc8ea5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:29:21.844 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:29:21.845 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:29:23.847 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:29:23.972 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:29:23.972+0000 7fc2145d2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:31:23.878 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:31:23.879 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:31:25.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:31:26.007 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:31:26.006+0000 7f5aa8937700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:33:25.912 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:33:25.914 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:33:27.915 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:33:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:33:28.042+0000 7f0c7294a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:35:27.947 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:35:27.948 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:35:29.949 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:35:30.075 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:35:30.075+0000 7fc3488f3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:37:29.980 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:37:29.981 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:37:31.982 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:37:32.108 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:37:32.108+0000 7f9bbcc47700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:39:32.014 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:39:32.015 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:39:34.018 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:39:34.144 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:39:34.143+0000 7f11d3deb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:41:34.049 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:41:34.050 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:41:36.051 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:41:36.177 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:41:36.177+0000 7f8a3594b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:43:36.083 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:43:36.083 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:43:38.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:43:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:43:38.211+0000 7f31a4d9d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:45:38.116 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:45:38.117 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:45:40.119 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:45:40.245 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:45:40.245+0000 7f9afa9f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:47:40.150 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:47:40.151 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:47:42.153 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:47:42.279 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:47:42.279+0000 7f55111d7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:49:42.184 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:49:42.185 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:49:44.185 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:49:44.311 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:49:44.311+0000 7f76aa50a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:51:44.217 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:51:44.218 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:51:46.219 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:51:46.344 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:51:46.344+0000 7f66b6c83700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:53:46.250 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:53:46.252 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:53:48.253 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:53:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:53:48.378+0000 7fa2ec1da700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:55:48.285 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:55:48.286 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:55:50.288 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:55:50.415 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:55:50.413+0000 7fde8d8e5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:57:50.320 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:57:50.321 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:57:52.323 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T14:57:52.449 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:57:52.447+0000 7fc639cd7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T14:59:52.355 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T14:59:52.356 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T14:59:54.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T14:59:54.483 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T14:59:54.480+0000 7f0efb3c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:01:54.388 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:01:54.399 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:01:56.400 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:01:56.527 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:01:56.524+0000 7fa8f494d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:03:56.433 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:03:56.434 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:03:58.435 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:03:58.562 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:03:58.558+0000 7f9c50198700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:05:58.467 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:05:58.468 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:06:00.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:06:00.594 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:06:00.590+0000 7f6014cc9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:08:00.500 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:08:00.526 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:08:02.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:08:02.654 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:08:02.650+0000 7fc997959700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:10:02.558 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:10:02.559 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:10:04.560 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:10:04.687 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:10:04.683+0000 7fae9d60d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:12:04.592 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:12:04.605 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:12:06.607 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:12:06.734 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:12:06.730+0000 7f666a08b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:14:06.639 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:14:06.656 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:14:08.658 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:14:08.785 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:14:08.780+0000 7f9e8e44d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:16:08.690 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:16:08.691 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:16:10.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:16:10.818 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:16:10.813+0000 7f6b2ef3f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:18:10.723 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:18:10.738 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:18:12.739 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:18:12.865 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:18:12.861+0000 7f7b13748700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:20:12.771 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:20:12.772 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:20:14.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:20:14.900 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:20:14.895+0000 7f0bf9900700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:22:14.806 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:22:14.807 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:22:16.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:22:16.935 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:22:16.930+0000 7f966497d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:24:16.840 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:24:16.841 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:24:18.842 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:24:18.968 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:24:18.963+0000 7fcf34f0b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:26:18.873 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:26:18.874 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:26:20.876 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:26:21.003 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:26:20.998+0000 7f196ae85700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:28:20.908 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:28:20.910 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:28:22.911 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:28:23.038 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:28:23.033+0000 7ff5974e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:30:22.943 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:30:22.944 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:30:24.945 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:30:25.071 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:30:25.066+0000 7f3f5c5d0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:32:24.976 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:32:24.977 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:32:26.979 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:32:27.105 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:32:27.100+0000 7fb9febb8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:34:27.010 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:34:27.011 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:34:29.012 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:34:29.139 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:34:29.134+0000 7fde994e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:36:29.044 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:36:29.045 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:36:31.046 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:36:31.172 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:36:31.167+0000 7f218306e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:38:31.077 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:38:31.078 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:38:33.080 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:38:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:38:33.201+0000 7ffa44a52700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:40:33.112 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:40:33.113 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:40:35.114 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:40:35.240 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:40:35.235+0000 7f625aa99700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:42:35.145 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:42:35.146 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:42:37.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:42:37.274 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:42:37.269+0000 7ff98942e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:44:37.179 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:44:37.180 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:44:39.181 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:44:39.307 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:44:39.302+0000 7f0b9f5f9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:46:39.212 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:46:39.214 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:46:41.215 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:46:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:46:41.336+0000 7fb6fdc4c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:48:41.246 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:48:41.247 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:48:43.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:48:43.374 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:48:43.369+0000 7f27c32e7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:50:43.279 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:50:43.280 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:50:45.282 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:50:45.408 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:50:45.403+0000 7fdfa78b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:52:45.314 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:52:45.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:52:47.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:52:47.443 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:52:47.438+0000 7efc5ee06700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:54:47.349 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:54:47.350 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:54:49.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:54:49.477 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:54:49.472+0000 7f78a8b73700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:56:49.382 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:56:49.383 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:56:51.384 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T15:56:51.510 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:56:51.505+0000 7f3ca950e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T15:58:51.416 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T15:58:51.418 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T15:58:53.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T15:58:53.545 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T15:58:53.540+0000 7f9b850ce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:00:53.451 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:00:53.453 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:00:55.454 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:00:55.581 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:00:55.576+0000 7f0b7a5f5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:02:55.487 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:02:55.487 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:02:57.488 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:02:57.616 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:02:57.611+0000 7f6f46707700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:04:57.525 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:04:57.527 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:04:59.528 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:04:59.655 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:04:59.649+0000 7fca53d41700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:06:59.561 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:06:59.563 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:07:01.564 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:07:01.693 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:07:01.686+0000 7f3226d74700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:09:01.597 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:09:01.598 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:09:03.600 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:09:03.728 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:09:03.723+0000 7f1cce03f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:11:03.634 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:11:03.635 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:11:05.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:11:05.769 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:11:05.763+0000 7f497cc52700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:13:05.674 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:13:05.675 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:13:07.677 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:13:07.805 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:13:07.799+0000 7f2c0361d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:15:07.710 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:15:07.712 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:15:09.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:15:09.842 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:15:09.836+0000 7f09a648d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:17:09.745 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:17:09.746 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:17:11.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:17:11.876 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:17:11.870+0000 7fa20d11e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:19:11.783 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:19:11.784 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:19:13.786 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:19:13.914 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:19:13.908+0000 7fc891038700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:21:13.819 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:21:13.821 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:21:15.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:21:15.952 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:21:15.946+0000 7effb20b7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:23:15.857 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:23:15.858 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:23:17.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:23:17.987 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:23:17.982+0000 7f1409aae700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:25:17.894 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:25:17.895 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:25:19.896 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:25:20.025 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:25:20.019+0000 7f402f5ca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:27:19.931 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:27:19.932 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:27:21.934 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:27:22.065 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:27:22.059+0000 7f79f4534700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:29:21.970 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:29:21.971 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:29:23.973 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:29:24.099 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:29:24.093+0000 7f269ccd6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:31:24.004 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:31:24.005 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:31:26.006 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:31:26.135 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:31:26.128+0000 7ff078ac7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:33:26.040 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:33:26.041 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:33:28.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:33:28.169 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:33:28.163+0000 7feb61390700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:35:28.074 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:35:28.075 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:35:30.076 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:35:30.204 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:35:30.198+0000 7f87d270a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:37:30.109 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:37:30.110 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:37:32.111 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:37:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:37:32.234+0000 7f5ab31c1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:39:32.145 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:39:32.147 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:39:34.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:39:34.276 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:39:34.270+0000 7f946ee42700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:41:34.181 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:41:34.181 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:41:36.183 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:41:36.311 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:41:36.306+0000 7f061dad9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:43:36.217 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:43:36.218 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:43:38.220 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:43:38.348 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:43:38.342+0000 7f593e10a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:45:38.253 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:45:38.254 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:45:40.256 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:45:40.386 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:45:40.380+0000 7fa199ed3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:47:40.290 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:47:40.291 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:47:42.292 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:47:42.420 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:47:42.415+0000 7f4171ef1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:49:42.326 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:49:42.327 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:49:44.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:49:44.454 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:49:44.449+0000 7f1d53587700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:51:44.359 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:51:44.360 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:51:46.361 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:51:46.489 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:51:46.484+0000 7f851a029700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:53:46.395 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:53:46.396 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:53:48.397 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:53:48.524 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:53:48.519+0000 7fcb4a36b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:55:48.430 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:55:48.431 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:55:50.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:55:50.561 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:55:50.556+0000 7f20d2293700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:57:50.467 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:57:50.468 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:57:52.470 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T16:57:52.598 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:57:52.593+0000 7f133577a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T16:59:52.503 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T16:59:52.504 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T16:59:54.505 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T16:59:54.634 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T16:59:54.629+0000 7f17395c0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:01:54.539 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:01:54.540 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:01:56.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:01:56.669 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:01:56.664+0000 7f18a4c91700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:03:56.575 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:03:56.576 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:03:58.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:03:58.703 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:03:58.698+0000 7f033dd12700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:05:58.608 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:05:58.609 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:06:00.611 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:06:00.741 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:06:00.736+0000 7f4741be5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:08:00.646 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:08:00.647 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:08:02.648 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:08:02.775 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:08:02.770+0000 7ff254eab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:10:02.680 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:10:02.681 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:10:04.682 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:10:04.810 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:10:04.805+0000 7fa73bba0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:12:04.716 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:12:04.718 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:12:06.720 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:12:06.846 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:12:06.841+0000 7efe632cc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:14:06.751 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:14:06.751 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:14:08.753 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:14:08.881 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:14:08.876+0000 7f00e20ee700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:16:08.786 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:16:08.787 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:16:10.788 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:16:10.915 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:16:10.910+0000 7f08dfb73700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:18:10.821 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:18:10.822 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:18:12.823 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:18:12.952 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:18:12.947+0000 7f500c0ab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:20:12.858 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:20:12.859 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:20:14.861 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:20:14.987 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:20:14.983+0000 7f9b4bdf8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:22:14.893 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:22:14.894 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:22:16.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:22:17.022 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:22:17.018+0000 7f87af42e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:24:16.927 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:24:16.928 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:24:18.929 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:24:19.058 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:24:19.054+0000 7f575334e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:26:18.963 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:26:18.964 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:26:20.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:26:21.094 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:26:21.090+0000 7f7ab45a8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:28:21.000 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:28:21.001 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:28:23.001 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:28:23.129 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:28:23.125+0000 7f2761081700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:30:23.034 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:30:23.036 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:30:25.037 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:30:25.163 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:30:25.159+0000 7f7de0917700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:32:25.068 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:32:25.069 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:32:27.070 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:32:27.198 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:32:27.194+0000 7fef7b7e6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:34:27.103 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:34:27.104 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:34:29.105 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:34:29.233 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:34:29.229+0000 7f94f0779700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:36:29.138 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:36:29.139 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:36:31.140 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:36:31.268 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:36:31.265+0000 7f1327673700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:38:31.174 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:38:31.175 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:38:33.177 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:38:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:38:33.300+0000 7f918ef7f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:40:33.209 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:40:33.210 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:40:35.211 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:40:35.338 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:40:35.334+0000 7f3c55f01700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:42:35.243 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:42:35.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:42:37.245 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:42:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:42:37.370+0000 7f1d8eec2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:44:37.279 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:44:37.280 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:44:39.281 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:44:39.409 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:44:39.405+0000 7fc6e9abc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:46:39.314 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:46:39.316 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:46:41.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:46:41.446 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:46:41.442+0000 7f455bd3a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:48:41.350 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:48:41.351 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:48:43.352 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:48:43.481 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:48:43.477+0000 7f3c30e8e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:50:43.386 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:50:43.387 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:50:45.388 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:50:45.515 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:50:45.511+0000 7ffacb343700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:52:45.420 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:52:45.421 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:52:47.422 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:52:47.550 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:52:47.546+0000 7f40be35b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:54:47.456 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:54:47.457 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:54:49.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:54:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:54:49.580+0000 7f2106ed6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:56:49.490 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:56:49.491 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:56:51.492 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T17:56:51.621 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:56:51.618+0000 7fdc5901f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T17:58:51.527 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T17:58:51.528 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T17:58:53.529 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T17:58:53.658 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T17:58:53.654+0000 7fa61372d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:00:53.563 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:00:53.563 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:00:55.564 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:00:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:00:55.690+0000 7f0203fcb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:02:55.597 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:02:55.599 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:02:57.600 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:02:57.729 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:02:57.726+0000 7f8fb905b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:04:57.635 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:04:57.637 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:04:59.638 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:04:59.763 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:04:59.760+0000 7f9915e6b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:06:59.669 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:06:59.670 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:07:01.671 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:07:01.799 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:07:01.796+0000 7f3de6028700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:09:01.704 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:09:01.705 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:09:03.707 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:09:03.835 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:09:03.831+0000 7f6ac874a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:11:03.740 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:11:03.741 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:11:05.743 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:11:05.870 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:11:05.866+0000 7f0d14177700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:13:05.775 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:13:05.776 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:13:07.778 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:13:07.904 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:13:07.901+0000 7fa4cb6a7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:15:07.809 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:15:07.810 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:15:09.811 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:15:09.941 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:15:09.937+0000 7eff05e85700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:17:09.846 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:17:09.847 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:17:11.848 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:17:11.975 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:17:11.971+0000 7f5e1e710700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:19:11.882 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:19:11.883 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:19:13.884 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:19:14.011 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:19:14.007+0000 7f678d67f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:21:13.916 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:21:13.917 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:21:15.918 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:21:16.047 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:21:16.043+0000 7fb5eb6ca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:23:15.953 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:23:15.954 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:23:17.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:23:18.085 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:23:18.081+0000 7f52c8377700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:25:17.990 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:25:17.992 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:25:19.994 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:25:20.120 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:25:20.116+0000 7f01b86eb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:27:20.025 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:27:20.027 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:27:22.028 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:27:22.156 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:27:22.152+0000 7f74789ce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:29:22.061 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:29:22.062 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:29:24.064 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:29:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:29:24.186+0000 7f9480ba3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:31:24.096 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:31:24.097 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:31:26.098 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:31:26.226 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:31:26.221+0000 7fd062850700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:33:26.131 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:33:26.132 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:33:28.133 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:33:28.260 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:33:28.255+0000 7fa45851b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:35:28.164 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:35:28.165 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:35:30.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:35:30.295 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:35:30.290+0000 7f59397b6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:37:30.200 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:37:30.201 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:37:32.203 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:37:32.332 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:37:32.327+0000 7f4a786ba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:39:32.237 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:39:32.238 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:39:34.240 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:39:34.372 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:39:34.363+0000 7fe411030700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:41:34.273 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:41:34.274 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:41:36.276 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:41:36.405 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:41:36.399+0000 7f0e0b19b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:43:36.313 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:43:36.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:43:38.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:43:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:43:38.445+0000 7ff734ae5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:45:38.355 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:45:38.357 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:45:40.358 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:45:40.487 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:45:40.482+0000 7f62f8052700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:47:40.392 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:47:40.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:47:42.394 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:47:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:47:42.517+0000 7f460eb38700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:49:42.428 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:49:42.429 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:49:44.430 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:49:44.557 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:49:44.552+0000 7f6e75d5c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:51:44.462 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:51:44.463 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:51:46.465 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:51:46.593 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:51:46.588+0000 7f153e825700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:53:46.499 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:53:46.500 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:53:48.500 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:53:48.627 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:53:48.622+0000 7f4bc0003700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:55:48.532 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:55:48.533 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:55:50.534 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:55:50.662 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:55:50.658+0000 7f2df1db9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:57:50.567 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:57:50.568 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:57:52.569 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T18:57:52.699 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:57:52.695+0000 7fac11a29700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T18:59:52.602 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T18:59:52.603 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T18:59:54.604 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T18:59:54.732 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T18:59:54.727+0000 7f095edf4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:01:54.638 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:01:54.639 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:01:56.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:01:56.768 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:01:56.764+0000 7eff939ed700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:03:56.673 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:03:56.674 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:03:58.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:03:58.803 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:03:58.799+0000 7fa839701700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:05:58.708 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:05:58.710 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:06:00.711 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:06:00.837 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:06:00.833+0000 7fd609d7a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:08:00.742 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:08:00.743 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:08:02.744 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:08:02.872 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:08:02.868+0000 7f4bda11e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:10:02.777 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:10:02.778 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:10:04.780 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:10:04.912 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:10:04.908+0000 7fd69f060700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:12:04.827 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:12:04.828 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:12:06.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:12:06.956 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:12:06.952+0000 7fbc99761700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:14:06.861 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:14:06.862 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:14:08.863 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:14:08.991 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:14:08.987+0000 7f87e027f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:16:08.896 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:16:08.897 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:16:10.899 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:16:11.026 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:16:11.022+0000 7fca2a3dc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:18:10.933 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:18:10.934 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:18:12.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:18:13.065 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:18:13.061+0000 7f8cc1f5e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:20:12.970 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:20:12.971 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:20:14.971 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:20:15.100 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:20:15.096+0000 7f246c90f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:22:15.006 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:22:15.007 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:22:17.009 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:22:17.135 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:22:17.131+0000 7ff35b258700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:24:17.040 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:24:17.041 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:24:19.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:24:19.169 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:24:19.165+0000 7fececf4b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:26:19.075 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:26:19.076 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:26:21.078 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:26:21.205 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:26:21.201+0000 7fc65f24d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:28:21.111 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:28:21.137 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:28:23.138 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:28:23.267 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:28:23.264+0000 7fd38da7d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:30:23.172 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:30:23.173 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:30:25.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:30:25.303 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:30:25.300+0000 7f338891c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:32:25.208 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:32:25.230 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:32:27.232 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:32:27.361 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:32:27.359+0000 7fe76811f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:34:27.266 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:34:28.432 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:34:30.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:34:30.564 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:34:30.561+0000 7fa88baad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:36:30.468 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:36:30.470 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:36:32.471 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:36:32.597 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:36:32.595+0000 7f8b8b9e3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:38:32.502 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:38:32.522 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:38:34.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:38:34.652 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:38:34.650+0000 7faba4257700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:40:34.557 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:40:34.558 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:40:36.560 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:40:36.688 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:40:36.686+0000 7f9550e48700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:42:36.592 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:42:36.610 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:42:38.610 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:42:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:42:38.735+0000 7f605cce2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:44:38.642 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:44:38.644 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:44:40.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:44:40.773 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:44:40.771+0000 7fd1c9cfb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:46:40.678 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:46:40.679 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:46:42.679 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:46:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:46:42.804+0000 7fa4a6603700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:48:42.711 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:48:42.712 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:48:44.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:48:44.842 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:48:44.840+0000 7f4e01804700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:50:44.747 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:50:44.748 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:50:46.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:50:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:50:46.874+0000 7f50124ea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:52:46.782 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:52:46.783 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:52:48.784 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:52:48.910 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:52:48.909+0000 7f8b563de700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:54:48.816 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:54:48.817 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:54:50.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:54:50.945 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:54:50.943+0000 7f65338fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:56:50.850 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:56:50.851 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:56:52.852 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T19:56:52.977 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:56:52.976+0000 7f6984e39700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T19:58:52.884 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T19:58:52.885 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T19:58:54.886 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T19:58:55.011 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T19:58:55.010+0000 7f0cf5464700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:00:54.917 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:00:54.918 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:00:56.919 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:00:57.046 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:00:57.044+0000 7f526798a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:02:56.951 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:02:56.953 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:02:58.954 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:02:59.081 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:02:59.079+0000 7f599d67b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:04:58.987 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:04:58.988 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:05:00.989 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:05:01.118 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:05:01.116+0000 7fdd6b080700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:07:01.023 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:07:01.024 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:07:03.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:07:03.152 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:07:03.150+0000 7ffa5883e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:09:03.058 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:09:03.059 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:09:05.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:09:05.187 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:09:05.185+0000 7f7bd40be700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:11:05.092 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:11:05.093 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:11:07.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:11:07.224 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:11:07.222+0000 7f5980cdf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:13:07.129 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:13:07.130 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:13:09.131 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:13:09.257 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:13:09.255+0000 7fc69a20a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:15:09.163 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:15:09.164 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:15:11.165 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:15:11.294 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:15:11.292+0000 7fae6765c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:17:11.199 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:17:11.200 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:17:13.201 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:17:13.327 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:17:13.325+0000 7fc5e3e62700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:19:13.234 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:19:13.235 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:19:15.237 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:19:15.363 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:19:15.360+0000 7fa78f381700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:21:15.269 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:21:15.270 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:21:17.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:21:17.398 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:21:17.396+0000 7f11d1f01700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:23:17.303 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:23:17.304 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:23:19.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:23:19.432 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:23:19.429+0000 7f5b55a02700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:25:19.338 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:25:19.339 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:25:21.341 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:25:21.467 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:25:21.465+0000 7f52589b5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:27:21.373 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:27:21.374 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:27:23.375 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:27:23.503 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:27:23.500+0000 7f4177e75700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:29:23.409 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:29:23.410 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:29:25.411 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:29:25.538 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:29:25.536+0000 7f306e147700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:31:25.445 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:31:25.446 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:31:27.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:31:27.583 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:31:27.581+0000 7fa44cbd9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:33:27.488 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:33:27.489 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:33:29.490 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:33:29.621 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:33:29.618+0000 7f7b6d35d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:35:29.528 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:35:29.529 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:35:31.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:35:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:35:31.662+0000 7febed4ac700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:37:31.571 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:37:31.572 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:37:33.574 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:37:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:37:33.699+0000 7f35cc160700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:39:33.608 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:39:33.615 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:39:35.617 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:39:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:39:35.745+0000 7f334144a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:41:35.651 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:41:35.653 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:41:37.654 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:41:37.779 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:41:37.777+0000 7fc754367700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:43:37.686 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:43:37.687 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:43:39.688 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:43:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:43:39.811+0000 7faa4e258700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:45:39.720 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:45:39.721 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:45:41.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:45:41.851 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:45:41.849+0000 7f668092c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:47:41.756 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:47:41.757 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:47:43.758 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:47:43.885 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:47:43.882+0000 7f4af9649700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:49:43.790 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:49:43.791 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:49:45.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:49:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:49:45.919+0000 7fefe59b5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:51:45.827 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:51:45.829 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:51:47.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:51:47.958 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:51:47.956+0000 7f6564916700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:53:47.863 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:53:47.864 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:53:49.866 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:53:49.994 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:53:49.991+0000 7f03f5d89700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:55:49.900 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:55:49.901 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:55:51.902 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:55:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:55:52.029+0000 7fc56952d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:57:51.936 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:57:51.937 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:57:53.939 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T20:57:54.067 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:57:54.065+0000 7fd2355ee700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T20:59:53.972 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T20:59:53.973 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T20:59:55.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T20:59:56.104 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T20:59:56.101+0000 7f03b9761700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:01:56.010 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:01:56.011 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:01:58.013 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:01:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:01:58.139+0000 7fd3da8ae700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:03:58.048 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:03:58.049 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:04:00.051 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:04:00.181 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:04:00.178+0000 7fa02157f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:06:00.085 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:06:00.086 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:06:02.088 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:06:02.217 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:06:02.214+0000 7f0f3d90b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:08:02.122 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:08:02.123 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:08:04.124 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:08:04.253 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:08:04.251+0000 7f6457525700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:10:04.158 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:10:04.159 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:10:06.161 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:10:06.291 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:10:06.288+0000 7f9ab3f4c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:12:06.196 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:12:06.197 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:12:08.198 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:12:08.325 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:12:08.323+0000 7f5fd8216700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:14:08.230 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:14:08.231 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:14:10.233 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:14:10.362 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:14:10.359+0000 7ff0ab019700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:16:10.268 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:16:10.269 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:16:12.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:16:12.399 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:16:12.397+0000 7fbcdb9ed700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:18:12.305 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:18:12.305 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:18:14.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:18:14.436 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:18:14.434+0000 7fb20ec56700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:20:14.340 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:20:14.341 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:20:16.342 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:20:16.471 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:20:16.469+0000 7f5d5da8c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:22:16.376 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:22:16.377 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:22:18.378 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:22:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:22:18.503+0000 7f8cfd000700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:24:18.411 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:24:18.412 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:24:20.412 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:24:20.539 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:24:20.536+0000 7f4bf8fb3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:26:20.444 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:26:20.445 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:26:22.447 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:26:22.576 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:26:22.573+0000 7f6a4873a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:28:22.481 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:28:22.482 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:28:24.484 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:28:24.612 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:28:24.609+0000 7f11dae03700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:30:24.521 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:30:24.522 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:30:26.524 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:30:26.653 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:30:26.650+0000 7fe56328f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:32:26.558 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:32:26.559 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:32:28.561 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:32:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:32:28.687+0000 7f2aeec97700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:34:28.595 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:34:28.597 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:34:30.598 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:34:30.727 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:34:30.724+0000 7f01f67ce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:36:30.632 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:36:30.635 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:36:32.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:36:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:36:32.762+0000 7ff0765ba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:38:32.670 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:38:32.671 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:38:34.672 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:38:34.800 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:38:34.798+0000 7f408040f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:40:34.706 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:40:34.708 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:40:36.710 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:40:36.839 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:40:36.836+0000 7f4fb720f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:42:36.744 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:42:36.745 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:42:38.746 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:42:38.874 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:42:38.871+0000 7f383dc1b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:44:38.779 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:44:38.780 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:44:40.782 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:44:40.911 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:44:40.908+0000 7f2a4bc68700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:46:40.816 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:46:40.817 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:46:42.819 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:46:42.946 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:46:42.943+0000 7fe2fd358700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:48:42.851 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:48:42.852 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:48:44.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:48:44.986 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:48:44.983+0000 7f430e54d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:50:44.891 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:50:44.893 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:50:46.894 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:50:47.020 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:50:47.017+0000 7f8cc7288700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:52:46.925 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:52:46.926 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:52:48.928 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:52:49.057 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:52:49.054+0000 7f87136c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:54:48.963 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:54:48.964 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:54:50.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:54:51.094 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:54:51.091+0000 7fc2133ca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:56:50.999 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:56:51.001 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:56:53.003 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T21:56:53.132 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:56:53.129+0000 7fcbe8fef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T21:58:53.037 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T21:58:53.039 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T21:58:55.040 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T21:58:55.168 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T21:58:55.165+0000 7f4bf68c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:00:55.073 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:00:55.074 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:00:57.076 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:00:57.204 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:00:57.201+0000 7f13ca3f8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:02:57.110 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:02:57.113 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:02:59.114 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:02:59.241 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:02:59.238+0000 7f8286285700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:04:59.146 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:04:59.147 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:05:01.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:05:01.278 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:05:01.275+0000 7fa299adf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:07:01.183 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:07:01.184 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:07:03.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:07:03.314 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:07:03.311+0000 7f12094d3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:09:03.220 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:09:03.222 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:09:05.223 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:09:05.350 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:09:05.347+0000 7f4137fae700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:11:05.255 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:11:05.256 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:11:07.257 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:11:07.386 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:11:07.383+0000 7f1049ee2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:13:07.291 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:13:07.292 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:13:09.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:13:09.422 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:13:09.418+0000 7feadf1b9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:15:09.328 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:15:09.329 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:15:11.330 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:15:11.459 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:15:11.456+0000 7fe8a1472700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:17:11.364 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:17:11.365 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:17:13.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:17:13.495 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:17:13.491+0000 7fb5d9630700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:19:13.401 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:19:13.402 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:19:15.404 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:19:15.530 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:19:15.527+0000 7f9ef20b3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:21:15.436 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:21:15.437 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:21:17.439 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:21:17.568 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:21:17.564+0000 7f2871723700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:23:17.473 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:23:17.473 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:23:19.474 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:23:19.601 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:23:19.598+0000 7f88e6950700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:25:19.506 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:25:19.507 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:25:21.509 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:25:21.638 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:25:21.634+0000 7ffb24a9b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:27:21.543 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:27:21.545 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:27:23.546 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:27:23.673 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:27:23.669+0000 7f995978a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:29:23.578 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:29:23.579 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:29:25.580 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:29:25.709 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:29:25.705+0000 7f2331111700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:31:25.614 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:31:25.615 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:31:27.617 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:31:27.747 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:31:27.743+0000 7f3c801e7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:33:27.652 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:33:27.653 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:33:29.655 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:33:29.784 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:33:29.780+0000 7f5e025d5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:35:29.689 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:35:29.690 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:35:31.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:35:31.819 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:35:31.815+0000 7f9351265700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:37:31.723 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:37:31.724 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:37:33.725 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:37:33.853 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:37:33.849+0000 7f702c69a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:39:33.758 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:39:33.759 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:39:35.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:39:35.893 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:39:35.887+0000 7efe618d9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:41:35.796 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:41:35.797 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:41:37.799 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:41:37.926 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:41:37.922+0000 7f6b04c81700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:43:37.832 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:43:37.833 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:43:39.834 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-06-27T22:43:39.961 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:43:39.957+0000 7f8643ae3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:45:39.866 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-06-27T22:45:39.868 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-06-27T22:45:41.869 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-06-27T22:45:41.994 INFO:teuthology.orchestra.run.smithi138.stderr:2023-06-27T22:45:41.991+0000 7fd1b1af4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-06-27T22:47:36.798 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-06-27T22:47:36.799 DEBUG:teuthology.task.console_log:Killing console logger for smithi138 2023-06-27T22:47:36.799 DEBUG:teuthology.task.console_log:Killing console logger for smithi159 2023-06-27T22:47:36.800 DEBUG:teuthology.exit:Finished running handlers